DOCSLIB.ORG

CS460/626 : Natural Language Processing/Speech, NLP and the Web

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
CS460/626 : Natural Language Processing/Speech, NLP and the Web CS460/626 : Natural Language Processing/Speech, NLP and the Web Lecture 24, 25, 26 Wordnet Pushpak Bhattacharyya CSE Dept., IIT Bombay 17th and 19th (morning and night), 2013 NLP Trinity Problem Semantics NLP Trinity Parsing Part of Speech Tagging Morph Analysis Marathi French HMM Hindi English Language CRF MEMM Algorithm NLP Layer Discourse and Corefernce Increased Semantics Extraction Complexity Of Processing Parsing Chunking POS tagging Morphology Background Classification of Words Word Content Function Word Word Verb Noun Adjective Adverb Prepo Conjun Pronoun Interjection sition ction NLP: Thy Name is Disambiguation A word can have multiple meanings and A meaning can have multiple words Word with multiple meanings Where there is a will, Where there is a will, There are hundreds of relatives Where there is a will There is a way There are hundreds of relatives A meaning can have multiple words Proverb “A cheat never prospers” Proverb: “A cheat never prospers but can get rich faster” WSD should be distinguished from structural ambiguity Correct groupings a must … Iran quake kills 87, 400 injured When it rains cats and dogs run for cover Should be distinguished from structural ambiguity Correct groupings a must … Iran quake kills 87, 400 injured When it rains, cats and dogs runs for cover When it rains cats and dogs, run for cover Groups of words (Multiwords) and names can be ambiguous Broken guitar for sale, no strings attached (Pun) Washington voted Washington to power pujaa ne pujaa ke liye phul todaa (Pujaa plucked flowers for worship) (deep world knowledge) The use of a shin bone is to locate furniture in dark room Stages of processing Phonetics and phonology Morphology Lexical Analysis Syntactic Analysis Semantic Analysis Pragmatics Discourse Example of WSD Operation, surgery, surgical operation, surgical procedure, surgical process -- (a medical procedure involving an incision with instruments; performed to repair damage or arrest disease in a living body; "they will schedule the operation as soon as an operating room is available"; "he died while undergoing surgery") TOPIC->(noun) surgery#1 Operation, military operation -- (activity by a military or naval force (as a maneuver or campaign); "it was a joint operation of the navy and air force") TOPIC->(noun) military#1, armed forces#1, armed services#1, military machine#1, war machine#1 Operation -- ((computer science) data processing in which the result is completely specified by a rule (especially the processing that results from a single instruction); "it can perform millions of operations per second") TOPIC->(noun) computer science#1, computing#1 mathematical process, mathematical operation, operation -- ((mathematics) calculation by mathematical methods; "the problems at the end of the chapter demonstrated the mathematical processes involved in the derivation"; "they were learning the basic operations of arithmetic") TOPIC->(noun) mathematics#1, math#1, maths#1 IS WSD NEEDED IN LARGE APPLICATIONS? Word ambiguitytopic drift in IR {case, container} Drifted topic due to inapplicable sense!!! Query word: “Madrid bomb blast case” {case, suit, lawsuit} Drifted topic due to expanded term!!! {suit, apparel} 50 Our observations 45 43.75 43.75 On error Percentages 40 Due to various Factors 35 CLEF 2007 31.25 30 Transliteration 25 Translation Disambiguation 25 Stemmer Dictionary Error Percentage Error 20 18.75 Ranking 15 12.5 10 6.25 5 0 0 Hindi-English Marathi-English How about WSD and MT? Zaheer Khan, the India fast भारत के तेज गदबाज, जहर खान, इंलड bowler, has been ruled out of the के खलाफ ृ ंखला के शेष के बाहर शासन remainder of the series against कया गया है. (ruled in the England. administrative sense??) He will return to India and will be वह भारत लौटने और बाएँ हाथ के तेज replaced by left-arm seamer RP गदबाज आरपी संह वारा तथापत Singh. कया जाएगा. जहर लॉस म पहले टेट के दौरान Zaheer picked up a hamstring (lifted??) injury during the first Test at हैमिंग चोट उठाया. Lord's. वह भारत क वेट इंडीज म हाल ह म He had been withdrawn from the एक सह (correct??) टखने क चोट के squad for India's recent Test series कारण टेट ृ ंखला के लए टम से वापस in the West Indies due to a right ले लया गया था. ankle injury. Wordnet Psycholinguistic Theory Human lexical memory for nouns as a hierarchy. Can canary sing? - Pretty fast response. Can canary fly? - Slower response. Does canary have skin? – Slowest response. Animal (can move, has skin) Bird (can fly) canary (can sing) Wordnet - a lexical reference system based on psycholinguistic theories of human lexical memory. Essential Resource for WSD: Wordnet Word Forms Word Meanings F1 F2 F3 … Fn (bank) (rely) (depend) M E 1 E1,2 1,3 E1,1 (embankme (bank) nt) M 2 E E2,2 2,… (bank) M E 3 3,2 E3,3 … … M m Em,n Wordnet: History The first wordnet in the world was for English developed at Princeton over 15 years. The Eurowordnet- linked structure of European language wordnets was built in 1998 over 3 years with funding from the EC as a a mission mode project. Wordnets for Hindi and Marathi being built at IIT Bombay are amongst the first IL wordnets. All these are proposed to be linked into the IndoWordnet which eventually will be linked to the English and the Euro wordnets. Basic Principle Words in natural languages are polysemous. However, when synonymous words are put together, a unique meaning often emerges. Use is made of Relational Semantics. Lexical and Semantic relations in wordnet 1. Synonymy 2. Hypernymy / Hyponymy 3. Antonymy 4. Meronymy / Holonymy 5. Gradation 6. Entailment 7. Troponymy 1, 3 and 5 are lexical (word to word), rest are semantic (synset to synset). WordNet Sub-Graph Hyponymy Dwelling,abode Hypernymy Meronymy kitchen Hyponymy bckyard bedroom M e r house,home Gloss o veranda A place that serves as the living n Hyponymy y quarters of one or mor efamilies m y study guestroom hermitage cottage Fundamental Design Question Syntagmatic vs. Paradigmatic relations? Psycholinguistics is the basis of the design. When we hear a word, many words come to our mind by association. For English, about half of the associated words are syntagmatically related and half are paradignatically related. For cat animal, mammal- paradigmatic mew, purr, furry- syntagmatic Stated Fundamental Application of Wordnet: Sense Disambiguation Determination of the correct sense of the word The crane ate the fish vs. The crane was used to lift the load bird vs. machine The problem of Sense tagging Given a corpora To Assign correct sense to the words. This is sense tagging. Needs Word Sense Disambiguation (WSD) Highly important for Question Answering, Machine Translation, Text Mining tasks. Classification of Words Word Content Function Word Word Verb Noun Adjective Adverb Prepo Conjun Pronoun Interjection sition ction Example of sense marking: its need एक_4187 नए शोध_1138 के अनुसार_3123 िजन लोग_1189 का सामािजक_43540 जीवन_125623 यत_48029 होता है उनके दमाग_16168 के एक_4187 हसे_120425 म अधक_42403 जगह_113368 होती है। (According to a new research, those people who have a busy social life, have larger space in a part of their brain). नेचर यूरोसाइंस म छपे एक_4187 शोध_1138 के अनुसार_3123 कई_4118 लोग_1189 के दमाग_16168 के कैन से पता_11431 चला क दमाग_16168 का एक_4187 हसा_120425 एमगडाला सामािजक_43540 यतताओं_1438 के साथ_328602 सामंजय_166 के लए थोड़ा_38861 बढ़_25368 जाता है। यह शोध_1138 58 लोग_1189 पर कया गया िजसम उनक उ_13159 और दमाग_16168 क साइज़ के आँकड़े_128065 लए गए। अमरक_413405 टम_14077 ने पाया_227806 क िजन लोग_1189 क सोशल नेटवकग अधक_42403 है उनके दमाग_16168 का एमगडाला वाला हसा_120425 बाक_130137 लोग_1189 क तुलना_म_38220 अधक_42403 बड़ा_426602 है। दमाग_16168 का एमगडाला वाला हसा_120425 भावनाओं_1912 और मानसक_42151 िथत_1652 से जुड़ा हु आ माना_212436 जाता है। Ambiguity of लोग (People) लोग, जन, लोक, जनमानस, पिलक - एक से अधक यित "लोग के हत म काम करना चाहए" (English synset) multitude, masses, mass, hoi_polloi, people, the_great_unwashed - the common people generally "separate the warriors from the mass" "power to the people" दुनया, दुनयाँ, संसार, वव, जगत, जहाँ, जहान, ज़माना, जमाना, लोक, दुनयावाले, दुनयाँवाले, लोग - संसार म रहने वाले लोग "महामा गाँधी का समान पूर दुनया करती है / म इस दुनया क परवाह नहं करता / आज क दुनया पैसे के पीछे भाग रह है" (English synset) populace, public, world - people in general considered as a whole "he is a hero in the eyes of the public” Basic Principle Words in natural languages are polysemous. However, when synonymous words are put together, a unique meaning often emerges. Use is made of Relational Semantics. Componential Semantics where each word is a bundle of semantic features (as in the Schankian Conceptual Dependency system or Lexical Componential Semantics) is to be examined as a viable alternative. Componential Semantics Consider cat and tiger. Decide on componential attributes. Furry Carnivorous Heavy Domesticable For cat (Y, Y, N, Y) For tiger (Y,Y,Y,N) Complete and correct Attributes are difficult to design. Semantic relations in wordnet 1. Synonymy 2. Hypernymy / Hyponymy 3. Antonymy 4. Meronymy / Holonymy 5. Gradation 6. Entailment 7. Troponymy 1, 3 and 5 are lexical (word to word), rest are semantic (synset to synset). Synset: the foundation (house) 1.
Load more

Recommended publications
  • The Generative Lexicon
    The Generative Lexicon James Pustejovsky" Computer Science Department Brandeis University In this paper, I will discuss four major topics relating to current research in lexical seman- tics: methodology, descriptive coverage, adequacy of the representation, and the computational usefulness of representations. In addressing these issues, I will discuss what I think are some of the central problems facing the lexical semantics community, and suggest ways of best ap- proaching these issues. Then, I will provide a method for the decomposition of lexical categories and outline a theory of lexical semantics embodying a notion of cocompositionality and type coercion, as well as several levels of semantic description, where the semantic load is spread more evenly throughout the lexicon. I argue that lexical decomposition is possible if it is per- formed generatively. Rather than assuming a fixed set of primitives, I will assume a fixed number of generative devices that can be seen as constructing semantic expressions. I develop a theory of Qualia Structure, a representation language for lexical items, which renders much lexical ambiguity in the lexicon unnecessary, while still explaining the systematic polysemy that words carry. Finally, I discuss how individual lexical structures can be integrated into the larger lexical knowledge base through a theory of lexical inheritance. This provides us with the necessary principles of global organization for the lexicon, enabling us to fully integrate our natural language lexicon into a conceptual whole. 1. Introduction I believe we have reached an interesting turning point in research, where linguistic studies can be informed by computational tools for lexicology as well as an appre- ciation of the computational complexity of large lexical databases.
    [Show full text]
  • 1. Introduction
    University of Groningen Specific language impairment in Dutch de Jong, Jan IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below. Document Version Publisher's PDF, also known as Version of record Publication date: 1999 Link to publication in University of Groningen/UMCG research database Citation for published version (APA): de Jong, J. (1999). Specific language impairment in Dutch: inflectional morphology and argument structure. s.n. Copyright Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons). Take-down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum. Download date: 25-09-2021 Specific Language Impairment in Dutch: Inflectional Morphology and Argument Structure Jan de Jong Copyright ©1999 by Jan de Jong Printed by Print Partners Ipskamp, Enschede Groningen Dissertations in Linguistics 28 ISSN 0928-0030 Specific Language Impairment in Dutch: Inflectional Morphology and Argument Structure Proefschrift ter verkrijging van het doctoraat in de letteren aan de Rijksuniversiteit Groningen op gezag van de Rector Magnificus, dr.
    [Show full text]
  • Machine Reading = Text Understanding! David Israel! What Is It to Understand a Text? !
    Machine Reading = Text Understanding! David Israel! What is it to Understand a Text? ! • To understand a text — starting with a single sentence — is to:! • determine its truth (perhaps along with the evidence for its truth)! ! • determine its truth-conditions (roughly: what must the world be like for it to be true)! • So one doesn’t have to check how the world actually is — that is whether the sentence is true! • calculate its entailments! • take — or at least determine — appropriate action in light of it (and some given preferences/goals/desires)! • translate it accurately into another language! • and what does accuracy come to?! • ground it in a cognitively plausible conceptual space! • …….! • These are not necessarily competing alternatives! ! For Contrast: What is it to Engage Intelligently in a Dialogue? ! • Respond appropriately to what your dialogue partner has just said (and done)! • Typically, taking into account the overall purpose(s) of the interaction and the current state of that interaction (the dialogue state)! • Compare and contrast dialogue between “equals” and between a human and a computer, trying to assist the human ! • Siri is a very simple example of the latter! A Little Ancient History of Dialogue Systems! STUDENT (Bobrow 1964) Question-Answering: Word problems in simple algebra! “If the number of customers Tom gets is twice the square of 20% of the number of advertisements he runs, and the number of advertisements he runs is 45, what is the number of customers Tom gets?” ! Overview of the method ! . 1 Map referential expressions to variables. ! ! . 2 Use regular expression templates to identify and transform mathematical expressions.
    [Show full text]
  • Introduction to Wordnet: an On-Line Lexical Database
    Introduction to WordNet: An On-line Lexical Database George A. Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller (Revised August 1993) WordNet is an on-line lexical reference system whose design is inspired by current psycholinguistic theories of human lexical memory. English nouns, verbs, and adjectives are organized into synonym sets, each representing one underlying lexical concept. Different relations link the synonym sets. Standard alphabetical procedures for organizing lexical information put together words that are spelled alike and scatter words with similar or related meanings haphazardly through the list. Unfortunately, there is no obvious alternative, no other simple way for lexicographers to keep track of what has been done or for readers to ®nd the word they are looking for. But a frequent objection to this solution is that ®nding things on an alphabetical list can be tedious and time-consuming. Many people who would like to refer to a dictionary decide not to bother with it because ®nding the information would interrupt their work and break their train of thought. In this age of computers, however, there is an answer to that complaint. One obvious reason to resort to on-line dictionariesÐlexical databases that can be read by computersÐis that computers can search such alphabetical lists much faster than people can. A dictionary entry can be available as soon as the target word is selected or typed into the keyboard. Moreover, since dictionaries are printed from tapes that are read by computers, it is a relatively simple matter to convert those tapes into the appropriate kind of lexical database.
    [Show full text]
  • Leveraging Morpho-Semantics for the Discovery of Relations in Chinese Wordnet
    Leveraging Morpho-semantics for the Discovery of Relations in Chinese Wordnet Shu-Kai Hsieh Yu-Yun Chang Graduate Institute of Linguistics Graduate Institute of Linguistics National Taiwan University National Taiwan University Taipei, Taiwan Taipei, Taiwan [email protected] [email protected] Abstract data from the web (Cimiano et al., 2005), Semantic relations of different types have but runs the risk of influenced by the web played an important role in wordnet, and have genre (Alain, 2010). been widely recognized in various fields. In re- To enrich the relations coverage in Chinese cent years, with the growing interests of con- structing semantic network in support of in- Wordnet (CWN), in this paper, we propose telligent systems, automatic semantic relation an in situ approach by exploiting the morph- discovery has become an urgent task. This semantic information. This method, simple paper aims to extract semantic relations re- lying on the in situ morpho-semantic struc- and straightforward as it seems, does not incur ture in Chinese which can dispense of an the difficulties associated with lexical gaps in outside source such as corpus or web data. cross-language mapping that any translation- Manual evaluation of thousands of word pairs shows that most relations can be successful based model would encounter; and it is also predicted. We believe that it can serve as a economic and complementary with previous valuable starting point in complementing with approaches in that we can dispense of an out- other approaches, which will hold promise for the robust lexical relations acquisition. side corpus resource. In what follows, Section 2 gives a brief sum- 1 Introduction mary of lexical semantic relations acquisition Semantic relations are at the core of WordNet- from two perspectives.
    [Show full text]
  • Automatic Labeling of Troponymy for Chinese Verbs
    Automatic labeling of troponymy for Chinese verbs 羅巧Ê Chiao-Shan Lo*+ s!蓉 Yi-Rung Chen+ [email protected] [email protected] 林芝Q Chih-Yu Lin+ 謝舒ñ Shu-Kai Hsieh*+ [email protected] [email protected] *Lab of Linguistic Ontology, Language Processing and e-Humanities, +Graduate School of English/Linguistics, National Taiwan Normal University Abstract 以同©^Æ與^Y語意關¶Ë而成的^Y知X«，如ñ語^² (Wordnet)、P語^ ² (EuroWordnet)I，已有E分的研v，^²的úË_已øv完善。ú¼ø同的目的，- 研b語言@¦已úË'規!K-文^Y²路 (Chinese Wordnet,CWN)，è(Ð供完t的 -文­YK^©@分。6而，(目MK-文^Y²路ûq-，1¼目M;要/¡(ºº$ 定來標記同©^ÆK間的語意關Â，因d這些標記KxÏ尚*T成可L應(K一定規!。 因d，,Ç文章y%針對動^K間的上下M^Y語意關 (Troponymy)，Ðú一.ê動標 記的¹法。我們希望藉1句法上y定的句型 (lexical syntactic pattern)，úË一個能 ê 動½取ú動^上下M的ûq。透N^©意$定原G的U0，P果o:，dûqê動½取ú 的動^上M^，cº率將近~分K七A。,研v盼能將,¹法應(¼c(|U-的-文^ ²ê動語意關Â標記，以Ê知X,體Kê動úË，2而能有H率的úË完善的-文^Y知 XÇ源。 關關關uuu^^^：-文^Y²路、語©關Âê動標記、動^^Y語© Abstract Synset and semantic relation based lexical knowledge base such as wordnet, have been well-studied and constructed in English and other European languages (EuroWordnet). The Chinese wordnet (CWN) has been launched by Academia Sinica basing on the similar paradigm. The synset that each word sense locates in CWN are manually labeled, how- ever, the lexical semantic relations among synsets are not fully constructed yet. In this present paper, we try to propose a lexical pattern-based algorithm which can automatically discover the semantic relations among verbs, especially the troponymy relation. There are many ways that the structure of a language can indicate the meaning of lexical items. For Chinese verbs, we identify two sets of lexical syntactic patterns denoting the concept of hypernymy-troponymy relation.
    [Show full text]
  • Part 4: Lexical Semantics 2
    Natural Language Processing Part 4: lexical semantics 2 Marco Maggini Tecnologie per l'elaborazione del linguaggio Lexical semantics • A lexicon generally has a highly structured form ▫ It stores the meanings and uses of each word ▫ It encodes the relations between words and meanings • A lexeme is the minimal unit represented in the lexicon ▫ It pairs a stem (the orthographic/phonological form chosen to represent words) with a symbolic form for meaning representation (sense) • A dictionary is a kind of lexicon where meanings are expressed through definitions and examples son noun a boy or man in relation to either or both of his parents. • a male offspring of an animal. • a male descendant : the sons of Adam. • ( the Son) (in Christian belief) the second person of the Trinity; Christ. • a man considered in relation to his native country or area : one of Nevada's most famous sons. • a man regarded as the product of a particular person, influence, or environment : sons of the French Revolution. • (also my son) used by an elder person as a form of address for a boy or young man : “You're on private land, son.” 3 Marco Maggini Tecnologie per l'elaborazione del linguaggio Lexicons & dictionaries • Definitions in dictionaries exploit words and they may be circular (a word definition uses words whose definitions exploit that word) right adj. 1. of, relating to, situated on, or being the side of the body which is away from the side on which the heart is mostly located 2. located nearer to the right hand than to the left 3.
    [Show full text]
  • Ontology and the Lexicon
    Ontology and the Lexicon Graeme Hirst Department of Computer Science, University of Toronto, Toronto M5S 3G4, Ontario, Canada; [email protected] Summary. A lexicon is a linguistic object and hence is not the same thing as an ontology, which is non-linguistic. Nonetheless, word senses are in many ways similar to ontological concepts and the relationships found between word senses resemble the relationships found between concepts. Although the arbitrary and semi-arbitrary distinctions made by natural lan- guages limit the degree to which these similarities can be exploited, a lexicon can nonetheless serve in the development of an ontology, especially in a technical domain. 1 Lexicons and lexical knowledge 1.1 Lexicons A lexicon is a list of words in a language—a vocabulary—along with some knowl- edge of how each word is used. A lexicon may be general or domain-specific; we might have, for example, a lexicon of several thousand common words of English or German, or a lexicon of the technical terms of dentistry in some language. The words that are of interest are usually open-class or content words, such as nouns, verbs, and adjectives, rather than closed-class or grammatical function words, such as articles, pronouns, and prepositions, whose behaviour is more tightly bound to the grammar of the language. A lexicon may also include multi-word expressions such as fixed phrases (by and large), phrasal verbs (tear apart), and other common expressions (merry Christmas!; teach someone ’s grandmother to suck eggs; Elvis has left the building). ! " Each word or phrase in a lexicon is described in a lexical entry; exactly what is included in each entry depends on the purpose of the particular lexicon.
    [Show full text]
  • Lexical Scoring System of Lexical Chain for Quranic Document Retrieval
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by UKM Journal Article Repository GEMA Online® Journal of Language Studies 59 Volume 18(2), May 2018 http://doi.org/10.17576/gema-2018-1802-05 Lexical Scoring System of Lexical Chain for Quranic Document Retrieval Hamed Zakeri Rad [email protected] Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia Sabrina Tiun [email protected] Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia Saidah Saad [email protected] Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia ABSTRACT An Information Retrieval (IR) system aims to extract information based on a query made by a user on a particular subject from an extensive collection of text. IR is a process through which information is retrieved by submitting a query by a user in the form of keywords or to match words. In the Al-Quran, verses of the same or comparable topics are scattered throughout the text in different chapters, and it is therefore difficult for users to remember the many keywords of the verses. Therefore, in such situations, retrieving information using semantically related words is useful. In well-composed documents, the semantic integrity of the text (coherence) exists between the words. Lexical cohesion is the results of chains of related words that contribute to the continuity of the lexical meaning found within the text are a direct result of text being about the same thing (i.e. topic, etc.). This indicates that using an IR system and lexical chains are a useful and appropriate method for representing documents with concepts rather than using terms in order to have successful retrieval based on semantic relations.
    [Show full text]
  • WN-EWN Relations
    A Bottom-up Comparative Study of EuroWordNet and WordNet 3.0 Lexical and Semantic Relations Maria Teresa Pazienzaa, Armando Stellatoa, Alexandra Tudoracheab a) AI Research Group, Dept. of Computer Science, b) Dept. of Cybernetics, Statistics and Economic Systems and Production University of Rome, Informatics, Academy of Economic Studies Bucharest Tor Vergata Calea Dorobanţilor 15-17, 010552, Via del Politecnico 1, 00133 Rome, Italy Bucharest, Romania {pazienza,stellato,tudorache}@info.uniroma2.it [email protected] Abstract The paper presents a comparative study of semantic and lexical relations defined and adopted in WordNet and EuroWordNet. This document describes the experimental observations achieved through the analysis of data from different WordNet versions and EuroWordNet distributions for different languages, during the development of JMWNL (Java Multilingual WordNet Library), an extensible multilingual library for accessing WordNet-like resources in different languages and formats. The goal of this work was to realize an operative mapping between the relations defined in the two lexical resources and to unify library access and content navigation methods for both WordNet and EuroWordNet. The analysis focused on similarities, differences, semantic overlaps or inclusions, factual misinterpretations and inconsistencies between the intended and practical use of each single relation defined in these two linguistic resources. The paper details with examples the produced mapping, discussing required operations which implied merging, extending or simply keeping separate the examined relations. This document describes the experimental observations 1. Introduction achieved through the analysis of data from different We introduce a comparative study of semantic and WordNet versions and EuroWordNet distributions for lexical relations defined and adopted in two renowned different languages, during the development of JMWNL.
    [Show full text]
  • University of Hagen at CLEF 2004: Indexing and Translating Concepts for the GIRT Task
    University of Hagen at CLEF 2004: Indexing and Translating Concepts for the GIRT Task Johannes Leveling, Sven Hartrumpf Intelligent Information and Communication Systems University of Hagen (FernUniversitat¨ in Hagen) 58084 Hagen, Germany {Johannes.Leveling,Sven.Hartrumpf}@fernuni-hagen.de Abstract This paper describes the work done at the University of Hagen for our participation at the German Indexing and Retrieval Test (GIRT) task of the CLEF 2004 evaluation campaign. We conducted both monolingual and bilingual information retrieval experiments. For monolin- gual experiments with the German document collection, the focus is on applying and compar- ing three indexing methods targeting full word forms, disambiguated concepts, and extended semantic networks. The bilingual experiments for retrieving English documents for German topics rely on translating and expanding query terms based on a ranking of semantically related English terms for a German concept. English translations are compiled from heterogeneous resources, including multilingual lexicons such as EuroWordNet and dictionaries available online. 1 Introduction This paper investigates the performance of different indexing methods and automated retrieval strategies for the second participation in the University of Hagen in the evaluation campaign of the Cross Language Evaluation Forum (CLEF). In 2003, retrieval strategies based on generating query variants for a natural language (NL) query were compared (Leveling, 2003). The result of our best experiment in 2003 with respect to mean average precision (MAP) was 0.2064 for a run using both topic title and description. Before presenting the different approaches and results for monolingual and bilingual retrieval experi- ments with the GIRT documents (German Indexing and Retrieval Test database, see Kluck and Gey (2001)) in separate sections, a short overview of the analysis and transformation of natural language queries and documents is given.
    [Show full text]
  • Learning Antonyms with Paraphrases and A
    LEARNING ANTONYMS WITH PARAPHRASES AND A MORPHOLOGY-AWARE NEURAL NETWORK Sneha Rajana A THESIS In Computer and Information Science Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering 2017 _______________________________ Prof. Chris Callison-Burch Supervisor of Thesis _______________________________ Prof. Lyle Ungar Thesis Reader _______________________________ Prof. Boon Thau-Loo Masters Chairperson Acknowledgements I would like to thank Prof. Chris Callison-Burch for not only supervising this thesis but for being an amazing advisor and for getting me interested in NLP research. I am thankful to Prof. Lyle Ungar for serving on the thesis committee and Prof. Boon Thau Loo and the CIS department of Penn Engineering for providing me with the en- couragement and necessary resources to carry out this project. Besides this, I would also like to thank Vered Shwartz, Bar-Ilan University, Israel for providing us with the preprocessed corpus and software for parts of this work and whose insightful advice and guidance led to the successful completion of this project. I am grateful to my fellow Masters students whose constant encouragement and support was invaluable. Finally, thank you to my family. I am who I am because of you. ii Abstract Recognizing and distinguishing antonyms from other types of semantic relations is a key part of language understanding systems and has widespread applications in Natural Language Processing tasks. In this study, we present two novel methods for identifying antonyms. In our first method, we use paraphrase pairs containing negation markers to derive antonym pairs.
    [Show full text]