DOCSLIB.ORG

Wiktionary and NLP: Improving Synonymy Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Wiktionary and NLP: Improving Synonymy Networks Wiktionary and NLP: Improving synonymy networks Emmanuel Navarro Franck Sajous Bruno Gaume IRIT, CNRS & CLLE-ERSS, CNRS & CLLE-ERSS & IRIT, CNRS & Université de Toulouse Université de Toulouse Université de Toulouse [email protected] [email protected] [email protected] Laurent Prévot Hsieh ShuKai Kuo Tzu-Yi LPL, CNRS & English Department Graduate Institute of Linguistics Université de Provence NTNU, Taiwan NTU, Taiwan [email protected] [email protected] [email protected] Pierre Magistry Huang Chu-Ren TIGP, CLCLP, Academia Sinica, Dept. of Chinese and Bilingual Studies GIL, NTU, Taiwan Hong Kong Poly U. , Hong Kong. [email protected] [email protected] Abstract to this difficult situation. Among them Wiktionary seems to be the perfect resource for building com- Wiktionary, a satellite of the Wikipedia putational mono-lingual and multi-lingual lexica. initiative, can be seen as a potential re- This paper focuses therefore on Wiktionary, how source for Natural Language Processing. to improve it, and on its exploitation for creating It requires however to be processed be- resources. fore being used efficiently as an NLP re- In next section, we present some relevant infor- source. After describing the relevant as- mation about Wiktionary. Section 3 presents the pects of Wiktionary for our purposes, we lexical graphs we are using and the way we build focus on its structural properties. Then, them. Then we pay some attention to evaluation we describe how we extracted synonymy (§4) before exploring some tracks of improvement networks from this resource. We pro- suggested by Wiktionary structure itself. vide an in-depth study of these synonymy networks and compare them to those ex- 2 Wiktionary As previously said, NLP suffers from a lack of tracted from traditional resources. Fi- lexical resources, be it due to the low-quality or nally, we describe two methods for semi- non-existence of such resources, or to copyrights- automatically improving this network by related problems. As an example, we consider adding missing relations: (i) using a kind French language resources. Jacquin et al. (2002) of semantic proximity measure; (ii) using highlighted the limitations and inconsistencies translation relations of Wiktionary itself. from the French EuroWordnet. Later, Sagot and Note: The experiments of this paper are based on Wik- Fišer (2008) explained how they needed to re- tionary’s dumps downloaded in year 2008. Differences may be observed with the current versions available online. course to PWN, BalkaNet (Tufis, 2000) and other resources (notably Wikipedia) to build WOLF, a 1 Introduction free French WordNet that is promising but still a Reliable and comprehensive lexical resources con- very preliminary resource. Some languages are stitute a crucial prerequisite for various NLP tasks. straight-off purely under-resourced. However their building cost keeps them rare. In The Web as Corpus initiative arose (Kilgarriff this context, the success of the Princeton Word- and Grefenstette, 2003) as an attempt to design Net (PWN) (Fellbaum, 1998) can be explained by tools and methodologies to use the web for over- the quality of the resource but also by the lack of coming data sparseness (Keller and Lapata, 2002). serious competitors. Widening this observation to Nevertheless, this initiative raised non-trivial tech- more languages only makes this observation more nical problems described in Baroni et al. (2008). acute. In spite of various initiatives, costs make Moreover, the web is not structured enough to eas- resource development extremely slow or/and re- ily and massively extract semantic relations. sult in non freely accessible resources. Collabo- In this context, Wiktionary could appear to be rative resources might bring an attractive solution a paradisiac playground for creating various lexi- 19 Proceedings of the 2009 Workshop on the People’s Web Meets NLP, ACL-IJCNLP 2009, pages 19–27, Suntec, Singapore, 7 August 2009. c 2009 ACL and AFNLP cal resources. We describe below the Wiktionary 2.2.2 Layouts resource and we explain the restrictions and prob- In the following paragraph, we outline wik- lems we are facing when trying to exploit it. This tionary’s general structure. We only consider description may complete few earlier ones, for ex- words in the wiktionary’s own language. ample Zesch et al. (2008a). An entry consists of a graphical form and a cor- responding article that is divided into the follow- 2.1 Collaborative editing ing, possibly embedded, sections: Wiktionary, the lexical companion to Wikipedia, etymology sections separate homonyms when • is a collaborative project to produce a free-content relevant; 1 multilingual dictionary. As the other Wikipedia’s among an etymology section, different parts • satellite projects, the resource is not experts-led, of speech may occur; rather filled by any kind of users. The might-be definitions and examples belong to a part of • inaccuracy of the resulting resource has lengthily speech section and may be subdivided into sub- been discussed and we will not debate it: see Giles senses; (2005) and Britannica (2006) for an illustration translations, synonyms/antonyms and hy- • of the controversy. Nevertheless, we think that pernyms/hyponyms are linked to a given part of Wiktionary should be less subject (so far) than speech, with or without subsenses distinctions. Wikipedia to voluntary misleading content (be it In figure 1 is depicted an article’s layout example. for ideological, commercial reasons, or alike). 2.2 Articles content As one may expect, a Wiktionary article2 may (not systematically) give information on a word’s part of speech, etymology, definitions, examples, pro- nunciation, translations, synonyms/antonyms, hy- pernyms/hyponyms, etc. 2.2.1 Multilingual aspects Wiktionary’s multilingual organisation may be surprising and not always meet one’s expectations or intuitions. Wiktionaries exist in 172 languages, but we can read on the English language main page, “1,248,097 entries with English definitions from over 295 languages”. Indeed, a given wik- tionary describes the words in its own language but also foreign words. For example, the English article moral includes the word in English (adjec- tive and noun) and Spanish (adjective and noun) Figure 1: Layout of boot article (shortened) but not in French. Another example, boucher, About subsenses, they are identified with an in- which does not exist in English, is an article of the dex when first introduced but they may appear as English wiktionary, dedicated to the French noun a plain text semantic feature (without index) when (a butcher) and French verb (to cork up). used in relations (translations, synonyms, etc.). It A given wiktionary’s ’in other languages’ left is therefore impossible to associate the relations menu’s links, point to articles in other wiktionar- arguments to subsenses. Secondly, subsense index ies describing the word in the current language. appears only in the current word (the source of the For example, the Français link in the dictionary relation) and not in the target word’s article it is article of the English wiktionary points to an arti- linked to (see orange French N. and Adj., Jan. 10, cle in the French one, describing the English word 20083). dictionary. A more serious issue appears when relations are shared by several parts of speech sections. In Ital- 1http://en.wiktionary.org/ 2What article refers to is more fuzzy than classical entry 3http://fr.wiktionary.org/w/index.php? or acceptance means. title=orange&oldid=2981313 20 ian, both synonyms and translations parts are com- lows in §3.2. When merging information extracted mon to all words categories (see for example car- from several languages, the homogenisation of the dinale N. and Adj., Apr. 26, 20094). data structure often leads to the choice of the poor- 2.3 Technical issues est one, resulting in a loss of information. As Wikipedia and the other Wikimedia Founda- 2.5 The bigger the better? tion’s projects, the Wiktionary’s content manage- Taking advantage of colleagues mastering various ment system relies on the MediaWiki software languages, we studied the wiktionary of the fol- and on the wikitext. As stated in Wikipedia’s lowing languages: French, English, German, Pol- MetaWiki article, “no formal syntax has been de- ish and Mandarin Chinese. A first remark con- fined” for the MediaWiki and consequently it is cerns the size of the resource. The official num- not possible to write a 100% reliable parser. ber of declared articles in a given wiktionary in- Unlike Wikipedia, no HTML dump is available cludes a great number of meta-articles which are and one has to parse the Wikicode. Wikicode not word entries As of April 2009, the French wik- is difficult to handle since wiki templates require tionary reaches the first rank6, before the English handwritten rules that need to be regularly up- one. This can be explained by the automated im- dated. Another difficulty is the language-specific port of public-domain dictionaries articles (Littré encoding of the information. Just to mention one, 1863 and Dictionnaire de l’Académie Française the target language of a translation link is iden- 1932-1935). Table 1 shows the ratio between the tified by a 2 or 3 letters ISO-639 code for most total number of articles and the “relevant” ones languages. However in the Polish wiktionary the (numbers based on year 2008 snapshots). complete name of the language name (angielski, Total Meta∗ Other∗∗ Relevant francuski, . ) is used. fr 728,266 25,244 369,948 337,074 46% en 905,963 46,202 667,430 192,331 21% 2.4 Parsing and modeling de 88,912 7,235 49,672 32,005 36% The (non-exhaustive) aforementioned list of diffi- pl 110,369 4,975 95,241 10,153 9% zh 131,752 8,195 112,520 1,037 0.7% culties (see §2.2.2 and §2.3) leads to the following ∗ templates definitions, help pages, user talks, etc.
Load more

Recommended publications
  • Dictionary Users Do Look up Frequent Words. a Logfile Analysis
    Erschienen in: Müller-Spitzer, Carolin (Hrsg.): Using Online Dictionaries. Berlin/ Boston: de Gruyter, 2014. (Lexicographica: Series Maior 145), S. 229-249. Alexander Koplenig, Peter Meyer, Carolin Müller-Spitzer Dictionary users do look up frequent words. A logfile analysis Abstract: In this paper, we use the 2012 log files of two German online dictionaries (Digital Dictionary of the German Language1 and the German Version of Wiktionary) and the 100,000 most frequent words in the Mannheim German Reference Corpus from 2009 to answer the question of whether dictionary users really do look up fre- quent words, first asked by de Schryver et al. (2006). By using an approach to the comparison of log files and corpus data which is completely different from that of the aforementioned authors, we provide empirical evidence that indicates - contra - ry to the results of de Schryver et al. and Verlinde/Binon (2010) - that the corpus frequency of a word can indeed be an important factor in determining what online dictionary users look up. Finally, we incorporate word dass Information readily available in Wiktionary into our analysis to improve our results considerably. Keywords: log file, frequency, corpus, headword list, monolingual dictionary, multi- lingual dictionary Alexander Koplenig: Institut für Deutsche Sprache, R 5, 6-13, 68161 Mannheim, +49-(0)621-1581- 435, [email protected] Peter Meyer: Institut für Deutsche Sprache, R 5, 6-13, 68161 Mannheim, +49-(0)621-1581-427, [email protected] Carolin Müller-Spitzer: Institut für Deutsche Sprache, R 5, 6-13, 68161 Mannheim, +49-(0)621-1581- 429, [email protected] Introduction We would like to Start this chapter by asking one of the most fundamental questions for any general lexicographical endeavour to describe the words of one (or more) language(s): which words should be included in a dictionary? At first glance, the answer seems rather simple (especially when the primary objective is to describe a language as completely as possible): it would be best to include every word in the dictionary.
    [Show full text]
  • Etytree: a Graphical and Interactive Etymology Dictionary Based on Wiktionary
    Etytree: A Graphical and Interactive Etymology Dictionary Based on Wiktionary Ester Pantaleo Vito Walter Anelli Wikimedia Foundation grantee Politecnico di Bari Italy Italy [email protected] [email protected] Tommaso Di Noia Gilles Sérasset Politecnico di Bari Univ. Grenoble Alpes, CNRS Italy Grenoble INP, LIG, F-38000 Grenoble, France [email protected] [email protected] ABSTRACT a new method1 that parses Etymology, Derived terms, De- We present etytree (from etymology + family tree): a scendants sections, the namespace for Reconstructed Terms, new on-line multilingual tool to extract and visualize et- and the etymtree template in Wiktionary. ymological relationships between words from the English With etytree, a RDF (Resource Description Framework) Wiktionary. A first version of etytree is available at http: lexical database of etymological relationships collecting all //tools.wmflabs.org/etytree/. the extracted relationships and lexical data attached to lex- With etytree users can search a word and interactively emes has also been released. The database consists of triples explore etymologically related words (ancestors, descendants, or data entities composed of subject-predicate-object where cognates) in many languages using a graphical interface. a possible statement can be (for example) a triple with a lex- The data is synchronised with the English Wiktionary dump eme as subject, a lexeme as object, and\derivesFrom"or\et- at every new release, and can be queried via SPARQL from a ymologicallyEquivalentTo" as predicate. The RDF database Virtuoso endpoint. has been exposed via a SPARQL endpoint and can be queried Etytree is the first graphical etymology dictionary, which at http://etytree-virtuoso.wmflabs.org/sparql.
    [Show full text]
  • User Contributions to Online Dictionaries Andrea Abel and Christian M
    The dynamics outside the paper: User Contributions to Online Dictionaries Andrea Abel and Christian M. Meyer Electronic lexicography in the 21st century (eLex): Thinking outside the paper, Tallinn, Estonia, October 17–19, 2013. 18.10.2013 | European Academy of Bozen/Bolzano and Technische Universität Darmstadt | Andrea Abel and Christian M. Meyer | 1 Introduction Online dictionaries rely increasingly on their users and leverage methods for facilitating user contributions at basically any step of the lexicographic process. 18.10.2013 | European Academy of Bozen/Bolzano and Technische Universität Darmstadt | Andrea Abel and Christian M. Meyer | 2 Previous work . Mostly focused on specific type/dimension of user contribution . e.g., Carr (1997), Storrer (1998, 2010), Køhler Simonsen (2005), Fuertes-Olivera (2009), Melchior (2012), Lew (2011, 2013) . Ambiguous, partly overlapping terms: www.wordle.net/show/wrdl/7124863/User_contributions (02.10.2013) www.wordle.net/show/wrdl/7124863/User_contributions http:// 18.10.2013 | European Academy of Bozen/Bolzano and Technische Universität Darmstadt | Andrea Abel and Christian M. Meyer | 3 Research goals and contribution How to effectively plan the user contributions for new and established dictionaries? . Comprehensive and systematic classification is still missing . Mann (2010): analysis of 88 online dictionaries . User contributions roughly categorized as direct / indirect contributions and exchange with other dictionary users . Little detail given, since outside the scope of the analysis . We use this as a basis for our work We propose a novel classification of the different types of user contributions based on many practical examples 18.10.2013 | European Academy of Bozen/Bolzano and Technische Universität Darmstadt | Andrea Abel and Christian M.
    [Show full text]
  • Wiktionary Matcher
    Wiktionary Matcher Jan Portisch1;2[0000−0001−5420−0663], Michael Hladik2[0000−0002−2204−3138], and Heiko Paulheim1[0000−0003−4386−8195] 1 Data and Web Science Group, University of Mannheim, Germany fjan, [email protected] 2 SAP SE Product Engineering Financial Services, Walldorf, Germany fjan.portisch, [email protected] Abstract. In this paper, we introduce Wiktionary Matcher, an ontology matching tool that exploits Wiktionary as external background knowl- edge source. Wiktionary is a large lexical knowledge resource that is collaboratively built online. Multiple current language versions of Wik- tionary are merged and used for monolingual ontology matching by ex- ploiting synonymy relations and for multilingual matching by exploiting the translations given in the resource. We show that Wiktionary can be used as external background knowledge source for the task of ontology matching with reasonable matching and runtime performance.3 Keywords: Ontology Matching · Ontology Alignment · External Re- sources · Background Knowledge · Wiktionary 1 Presentation of the System 1.1 State, Purpose, General Statement The Wiktionary Matcher is an element-level, label-based matcher which uses an online lexical resource, namely Wiktionary. The latter is "[a] collaborative project run by the Wikimedia Foundation to produce a free and complete dic- tionary in every language"4. The dictionary is organized similarly to Wikipedia: Everybody can contribute to the project and the content is reviewed in a com- munity process. Compared to WordNet [4], Wiktionary is significantly larger and also available in other languages than English. This matcher uses DBnary [15], an RDF version of Wiktionary that is publicly available5. The DBnary data set makes use of an extended LEMON model [11] to describe the data.
    [Show full text]
  • Extracting an Etymological Database from Wiktionary Benoît Sagot
    Extracting an Etymological Database from Wiktionary Benoît Sagot To cite this version: Benoît Sagot. Extracting an Etymological Database from Wiktionary. Electronic Lexicography in the 21st century (eLex 2017), Sep 2017, Leiden, Netherlands. pp.716-728. hal-01592061 HAL Id: hal-01592061 https://hal.inria.fr/hal-01592061 Submitted on 22 Sep 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Extracting an Etymological Database from Wiktionary Benoît Sagot Inria 2 rue Simone Iff, 75012 Paris, France E-mail: [email protected] Abstract Electronic lexical resources almost never contain etymological information. The availability of such information, if properly formalised, could open up the possibility of developing automatic tools targeted towards historical and comparative linguistics, as well as significantly improving the automatic processing of ancient languages. We describe here the process we implemented for extracting etymological data from the etymological notices found in Wiktionary. We have produced a multilingual database of nearly one million lexemes and a database of more than half a million etymological relations between lexemes. Keywords: Lexical resource development; etymology; Wiktionary 1. Introduction Electronic lexical resources used in the fields of natural language processing and com- putational linguistics are almost exclusively synchronic resources; they mostly include information about inflectional, derivational, syntactic, semantic or even pragmatic prop- erties of their entries.
    [Show full text]
  • Wiktionary Matcher Results for OAEI 2020
    Wiktionary Matcher Results for OAEI 2020 Jan Portisch1;2[0000−0001−5420−0663] and Heiko Paulheim1[0000−0003−4386−8195] 1 Data and Web Science Group, University of Mannheim, Germany fjan, [email protected] 2 SAP SE Product Engineering Financial Services, Walldorf, Germany [email protected] Abstract. This paper presents the results of the Wiktionary Matcher in the Ontology Alignment Evaluation Initiative (OAEI) 2020. Wiktionary Matcher is an ontology matching tool that exploits Wiktionary as exter- nal background knowledge source. Wiktionary is a large lexical knowl- edge resource that is collaboratively built online. Multiple current lan- guage versions of Wiktionary are merged and used for monolingual on- tology matching by exploiting synonymy relations and for multilingual matching by exploiting the translations given in the resource. This is the second OAEI participation of the matching system. Wiktionary Matcher has been improved and is the best performing system on the knowledge graph track this year.3 Keywords: Ontology Matching · Ontology Alignment · External Re- sources · Background Knowledge · Wiktionary 1 Presentation of the System 1.1 State, Purpose, General Statement The Wiktionary Matcher is an element-level, label-based matcher which uses an online lexical resource, namely Wiktionary. The latter is "[a] collaborative project run by the Wikimedia Foundation to produce a free and complete dic- tionary in every language"4. The dictionary is organized similarly to Wikipedia: Everybody can contribute to the project and the content is reviewed in a com- munity process. Compared to WordNet [2], Wiktionary is significantly larger and also available in other languages than English. This matcher uses DBnary [13], an RDF version of Wiktionary that is publicly available5.
    [Show full text]
  • S Wiktionary Wikisource Wikibooks Wikiquote Wikimedia Commons
    SCHWESTERPROJEKTE Wiktionary S Das Wiktionary ist der lexikalische Partner der freien Enzyklopädie Wikipedia: ein Projekt zur Erstellung freier Wörterbücher und Thesau- ri. Während die Wikipedia inhaltliche Konzepte beschreibt, geht es in ihrem ältesten Schwester- projekt, dem 2002 gegründeten Wiktionary um Wörter, ihre Grammatik und Etymologie, Homo- nyme und Synonyme und Übersetzungen. Wikisource Wikisource ist eine Sammlung von Texten, die entweder urheberrechtsfrei sind oder unter ei- ner freien Lizenz stehen. Das Projekt wurde am 24. November 2003 gestartet. Der Wiktionary-EIntrag zum Wort Schnee: Das Wörterbuch präsen- Zunächst mehrsprachig auf einer gemeinsamen tiert Bedeutung, Deklination, Synonyme und Übersetzungen. Plattform angelegt, wurde es später in einzel- ne Sprachversionen aufgesplittet. Das deutsche Teilprojekt zählte im März 2006 über 2000 Texte Wikisource-Mitarbeiter arbeiten an einer digitalen, korrekturge- und über 100 registrierte Benutzer. lesenen und annotierten Ausgabe der Zimmerischen Chronik. Wikibooks Das im Juli 2003 aus der Taufe gehobene Projekt Wikibooks dient der gemeinschaftlichen Schaf- fung freier Lehrmaterialien – vom Schulbuch über den Sprachkurs bis zum praktischen Klet- terhandbuch oder der Go-Spielanleitung Wikiquote Wikiquote zielt darauf ab, auf Wiki-Basis ein freies Kompendium von Zitaten und Das Wikibooks-Handbuch Go enthält eine ausführliche Spielanleitung Sprichwörtern in jeder Sprache zu schaffen. Die des japanischen Strategiespiels. Artikel über Zitate bieten (soweit bekannt) eine Quellenangabe und werden gegebenenfalls in die deutsche Sprache übersetzt. Für zusätzliche Das Wikimedia-Projekt Wikiquote sammelt Sprichwörter und Informationen sorgen Links in die Wikipedia. Zitate, hier die Seite zum Schauspieler Woody Allen Wikimedia Commons Wikimedia Commons wurde im September 2004 zur zentralen Aufbewahrung von Multime- dia-Material – Bilder, Videos, Musik – für alle Wi- kimedia-Projekte gegründet.
    [Show full text]
  • Thank You - Wiktionary 4/24/11 9:56 PM Thank You
    thank you - Wiktionary 4/24/11 9:56 PM thank you Definition from Wiktionary, the free dictionary See also thankyou and thank-you Contents 1 English 1.1 Etymology 1.2 Pronunciation 1.3 Interjection 1.3.1 Synonyms 1.3.2 Translations 1.4 Noun 1.5 References 1.6 See also English Etymology Thank you is a shortened expression for I thank you; it is attested since c. 1400.[1] Pronunciation Audio (UK) (file) Interjection thank you 1. An expression of gratitude or politeness, in response to something done or given. http://en.wiktionary.org/wiki/thank_you Page 1 of 5 thank you - Wiktionary 4/24/11 9:56 PM Synonyms cheers (informal), thanks, thanks very much, thank you very much, thanks a lot, ta (UK, Australia), thanks a bunch (informal), thanks a million (informal), much obliged Translations an expression of gratitude [hide ▲] Select targeted languages ﺳﻮﭘﺎﺱ ,Afar: gadda ge Kurdish: spas Afrikaans: dankie, baie dankie Ladin: giulan Albanian: faleminderit, ju falem nderit Lao: (kööp cai) Gheg: falimineres Latin: benignē dīcis (la), tibi gratiās agō (la) Latvian: paldies (lv) Aleut: qaĝaasakuq (Atkan), qaĝaalakux̂ Lithuanian: ačiū (lt) (Eastern) Luo: erokamano Alutiiq: quyanaa Macedonian: благодарам (mk) (blagódaram) American Sign Language: OpenB@Chin- (formal), фала (mk) (fála) (informal) PalmBack OpenB@FromChin-PalmUp Malagasy: misaotra (mg) Amharic: (am) (amesegenallo) Malay: terima kasih (ms) (ar) (mersii) Malayalam: (ml) (nandhi) ﻣﻴﺮﺳﻲ ,(ar) (shúkran) ﺷﻜﺮًﺍ :Arabic (informal) Maltese: grazzi (mt) Armenian: Maori: kia ora (mi) շնորհակալություն
    [Show full text]
  • Detecting Non-Compositional MWE Components Using Wiktionary
    Detecting Non-compositional MWE Components using Wiktionary Bahar Salehi,♣♥ Paul Cook♥ and Timothy Baldwin♣♥ NICTA Victoria Research Laboratory Department♣ of Computing and Information Systems ♥ The University of Melbourne Victoria 3010, Australia [email protected], [email protected], [email protected] Abstract components of a given MWE have a composi- tional usage. Experiments over two widely-used We propose a simple unsupervised ap- datasets show that our approach outperforms state- proach to detecting non-compositional of-the-art methods. components in multiword expressions based on Wiktionary. The approach makes 2 Related Work use of the definitions, synonyms and trans- Previous studies which have considered MWE lations in Wiktionary, and is applicable to compositionality have focused on either the iden- any type of MWE in any language, assum- tification of non-compositional MWE token in- ing the MWE is contained in Wiktionary. stances (Kim and Baldwin, 2007; Fazly et al., Our experiments show that the proposed 2009; Forthergill and Baldwin, 2011; Muzny and approach achieves higher F-score than Zettlemoyer, 2013), or the prediction of the com- state-of-the-art methods. positionality of MWE types (Reddy et al., 2011; 1 Introduction Salehi and Cook, 2013; Salehi et al., 2014). The identification of non-compositional MWE tokens A multiword expression (MWE) is a combina- is an important task when a word combination tion of words with lexical, syntactic or seman- such as kick the bucket or saw logs is ambiguous tic idiosyncrasy (Sag et al., 2002; Baldwin and between a compositional (generally non-MWE) Kim, 2009). An MWE is considered (semanti- and non-compositional MWE usage.
    [Show full text]
  • The Wiki Family of Web Sites
    The Federal Lawyer In Cyberia MICHAEL J. TONSING The Wiki Family of Web Sites he growth in the number of informative Web set of resources. Much of what follows is drawn from the sites seems exponential. It is increasingly hard self-descriptions on the Wiki Web sites themselves. Tto keep up with them. If you’re not using the “Wiki” family of sites, you’re missing some sources that Wiktionary are almost stupefying in their scope. A “wiki” is essen- Wiktionary (en.wikipedia.org/wiki/Wiktionary) is tially a Web site in which contributors can add their a sister site to Wikipedia. Wiktionary is not an online own copy. (I capitalize the word in most instances in encyclopedia but an online dictionary, and its entries this column to make it clear that I am referring to a par- are about words. A Wiktionary entry focuses on mat- ticular family of wikis. There are many other wikis out ters of language and wordsmithery, spelling, pronun- there. You may decide to start your own someday.) ciation, etymology, translation, usage, quotations, and links to related words and concepts. Because Wiktion- Wikipedia ary is not written on paper, it has no size limits, it can Wikipedia (www.wikipedia.org), then, is an online include links, and its information can be more timely encyclopedia in which visitors can read infor- than that found in a written dictionary. mation on the topics they visit, then edit the information on the site itself if they choose Wikisource to do so. (The name “Wikipedia” is a meld- Wikisource (en.wikipedia.org/wiki/Wikisource), which ing of the works “wiki” and “encyclopedia.”) dubs itself “the free library,” collects and stores previ- Out of what might at first have seemed like ously published texts in digital format; contents include online chaos has come semirespectability and novels, nonfiction works, letters, speeches, constitutional order.
    [Show full text]
  • The Wiki-Fication of the Dictionary: Defining Lexicography in the Digital Age
    THE WIKI-FICATION OF THE DICTIONARY: DEFINING LEXICOGRAPHY IN THE DIGITAL AGE Darrell J. Penta College of Professional Studies Northeastern University [email protected] Abstract The future of lexical reference books, such as the 20-volume Oxford English Dictionary (OED), is going to be determined, in part, by the emergence of free on-line dictionaries, such as Wiktionary and Urban Dictionary. Specifically, we are witnessing a paradigmatic shift of authority in which users, rather than editorial boards, are making decisions concerning the content associated with a lexical entry’s definition. In effect, an exclusive privilege formerly enjoyed by professional lexicographers is now being extended unequivocally to laypersons. It is pertinent to ask, therefore, what effect this state of affairs will have on the ways that dictionaries are compiled and used. For some, including Jill Lepore of the New Yorker Magazine, online collaborative lexical references are “Maoist” resources, “cobble[d]…together” by non-experts who “pilfer” definitions (79). This paper rejects such a characterization and seeks, instead, to provide a description more suitable for critical inquiry. By contrasting the entry “bomb” as it appears in the OED, Wiktionary, and Urban Dictionary, and by making use of contemporary linguistic theory, the author posits that: word meanings are highly constrained by popular usage; and, users regularly provide semantically and pragmatically significant, and grammatical accurate, definitions; and, in providing users the flexibility to modify entries in real-time, user-generated dictionaries are uniquely practical as catalogues of the current state of language. It is concluded that, whereas traditional dictionaries may be the better resource for diachronic analyses of words, Wiktionary and the like may prove better for synchronic analyses.
    [Show full text]
  • SLIDE – a Sentiment Lexicon of Common Idioms
    SLIDE – a Sentiment Lexicon of Common Idioms Charles Jochim, Francesca Bonin, Roy Bar-Haim, Noam Slonim IBM Research fcharlesj,[email protected], froybar,[email protected] Abstract Idiomatic expressions are problematic for most sentiment analysis approaches, which rely on words as the basic linguistic unit. Compositional solutions for phrase sentiment are not able to handle idioms correctly because their sentiment is not derived from the sentiment of the individual words. Previous work has explored the importance of idioms for sentiment analysis, but has not addressed the breadth of idiomatic expressions in English. In this paper we present an approach for collecting sentiment annotation of idiomatic multiword expressions using crowdsourcing. We collect 10 annotations for each idiom and the aggregated label is shown to have good agreement with expert annotations. We describe the resulting publicly available lexicon and how it captures sentiment strength and ambiguity. The Sentiment Lexicon of IDiomatic Expressions (SLIDE) is much larger than previous idiom lexicons. The lexicon includes 5,000 frequently occurring idioms, as estimated from a large English corpus. The idioms were selected from Wiktionary, and over 40% of them were labeled as sentiment-bearing. Keywords: Idiom, Lexicon, Sentiment Analysis 1. Introduction of magnitude larger than previous idiom sentiment lexi- cons and focuses specifically on the most frequently used Multiword expressions (MWE) are a key challenge in Natu- idioms.1 In creating this resource, we are somewhat ag- ral Language Processing (Sag et al., 2002). Among MWEs, nostic to the exact definition of idiom. We are more gener- idioms are often defined as non-compositional multiword ally interested in sentiment analysis that can handle MWEs.
    [Show full text]