DOCSLIB.ORG

Distributional Semantics Meets Construction Grammar. Towards A

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Distributional Semantics Meets Construction Grammar. Towards A Distributional Semantics Meets Construction Grammar. Towards a Unified Usage-Based Model of Grammar and Meaning Giulia Rambelli Emmanuele Chersoni University of Pisa The Hong Kong Polytechnic University [email protected]@gmail.com Philippe Blache Chu-Ren Huang Aix-Marseille University The Hong Kong Polytechnic University [email protected] [email protected] Alessandro Lenci University of Pisa [email protected] Abstract Obj1 Obj2]). It is worth stressing that, even if the concept of construction is based on the idea In this paper, we propose a new type of seman- that linguistic properties actually emerge from lan- tic representation of Construction Grammar guage use, CxG theories have typically preferred that combines constructions with the vector to model the semantic content of constructions in representations used in Distributional Seman- tics. We introduce a new framework, Distribu- terms of hand-made, formal representations like tional Construction Grammar, where grammar those of Frame Semantics (Baker et al., 1998). and meaning are systematically modeled from This leaves open the issue of how semantic repre- language use, and finally, we discuss the kind sentations can be learned from empirical evidence, of contributions that distributional models can and how do they relate to the usage-based nature of provide to CxG representation from a linguis- Cxs. In fact, for a usage-based model of grammar tic and cognitive perspective. based on a strong syntax-semantics parallelism, it would be desirable to be grounded on a framework 1 Introduction allowing to learn the semantic content of Cxs from In the last decades, usage-based models of lan- language use. guage have captured the attention of linguistics In this perspective, a promising solution for and cognitive science (Tommasello, 2003; Bybee, representing constructional semantics is given by 2010). The different approaches covered by this an approach to meaning representations that has label are based on the assumptions that linguistic gained a rising interest in both computational lin- knowledge is embodied in mental processing and guistics and cognitive science, namely Distribu- representations that are sensitive to context and tional Semantics (henceforth DS). DS is a usage- statistical probabilities (Boyland, 2009), and that based model of word meaning, based on the well- language structures at all levels, from morphology established assumption that the statistical distribu- to syntax, emerge out of facts of actual language tion of linguistic items in context plays a key role usage (Bybee, 2010). in characterizing their semantic behaviour (Distri- A usage-based framework that turned out to be butional Hypothesis (Harris, 1954)). More pre- extremely influential is Construction Grammar cisely, Distributional Semantic Models (DSMs) (CxG) (Hoffman and Trousdale, 2013), a family represent the lexicon in terms of vector spaces, of theories sharing the fundamental idea that lan- where a lexical target is described in terms of a guage is a collection of form-meaning pairings vector (also known as embedding) built by identi- called constructions (henceforth Cxs)(Fillmore, fying in a corpus its syntactic and lexical contexts 1988; Goldberg, 2006). Cxs differ for their degree (Lenci, 2018). Lately, neural models to learn dis- of schematicity, ranging from morphemes (e.g., tributional vectors have gained massive popular- pre-, -ing), to complex words (e.g., daredevil) to ity: these algorithms build low-dimensional vec- filled or partially-filled idioms (e.g., give the devil tor representations by learning to optimally predict his dues or Jog (someones) memory) to more ab- the contexts of the target words (Mikolov et al., stract patterns like the ditransitive Cxs [Subj V 2013). On the negative side, DS lacks a clear 110 Proceedings of the First International Workshop on Designing Meaning Representations, pages 110–120 Florence, Italy, August 1st, 2019 c 2019 Association for Computational Linguistics connection with usage-based theoretical frame- has never formulated a systematic proposal for de- works. To the best of our knowledge, existing riving representations of constructional meaning attempts of linking DS with models of grammar from corpus data. Previous literature has mostly have rather targeted formal theories like Montague focused either on the automatic identification of Grammar and Categorial Grammar (Baroni et al., constructions on the basis of their formal features, 2014; Grefenstette and Sadrzadeh, 2015). or on modeling the meaning of a specific CxG. To sum up, both CxG and DS share the assump- For the former approach, we should mention tion that linguistic structures naturally emerge the works of Dunn(2017, 2019) that aim at au- from language usage, and that a representation tomatically inducing a set of grammatical units of both form and meaning of any linguistic item (Cxs) from a large corpus. On the one hand, can be modeled through its distributional statistics, Dunn’s contributions provide a method for extract- and more generally, with the quantitative informa- ing Cxs from corpora, but on the other hand they tion derived from corpus data. However, these two are mainly concerned with the formal side of the models still live in parallel worlds. On the one constructions, and especially with the problem of hand, CxG is a model of grammar in search for how syntactic constraints are learned. Some sort a consistent usage-based model of meaning, and, of semantic representation is included, in the form conversely, DS is a computational framework to of semantic cluster of word embeddings to which build semantic representations in search for an em- the word forms appearing in the constructions are pirically adequate theory of grammar. assigned. However, these works do not present As we illustrate in Section 2, occasional en- any evaluation of the construction representations counters between DS and CxG have already hap- in terms of semantic tasks. pened, but we believe that new fruitful advances Another line of research has focused in using could come from the exploitation of the mu- constructions for building computational models tual synergies between CxG and DS, and by let- of language acquisition. Alishahi and Stevenson ting these two worlds finally meet and interact (2008) propose a model for the representation, ac- in a more systematic way. Following this direc- quisition and use of verb argument structure by tion of research, we introduce a new representa- formulating constructions as probabilistic associ- tion framework called Distributional Construc- ations between syntactic and semantic properties tion Grammar, which aims at bringing together of verbs and their arguments. This probabilistic these two theoretical paradigms. Our goal is to association emerges over time through a Bayesian integrate distributional information into construc- acquisition process in which similar verb usages tions by completing their semantic structures with are detected and grouped together to form general distributional vectors extracted from large textual constructions, based on their syntactic and seman- corpora, as samples of language usage. tic properties. Despite the success of this model, These pages are structured as follows: after the semantic representation of argument structure reviewing existing literature on CxG and related is still symbolic and each semantic category of in- computational studies, in Section3 we outline put constructions are manually compiled, in con- the key characteristics of our theoretical proposal, trast with the usage-based nature of constructions. while Section4 provides a general discussion Other studies used DSMs to model construc- about what contributions DSMs can provide to tional meaning, by focusing on a specific type of CxG representation from a linguistic and cognitive Cx rather than on the entire grammar. For exam- perspective. Although this is essentially a theoret- ple, Levshina and Heylen(2014) build a vector ical contribution, we outline ongoing work focus- space to study Dutch causative constructions with ing on its computational implementation and em- doen (‘do’) and laten (‘let’). They compute several pirical validation. We conclude by reporting future vector spaces with different context types, both for perspectives of research. the nouns that fill the Causer and Causee slot and for the verbs that fill the Effected Predicate slot. 2 Related Work Then, they cluster these nouns and verbs at differ- Despite the popularity of the constructional ap- ent levels of granularity and test which classifica- proach in corpus linguistics (Gries and Stefanow- tion better predicts the use of laten and doen. itsch, 2004), computational semantics research A recent trend in diachronic linguistics investi- 111 gates linguistic change as a sequence of gradual for some alternations, verb embeddings encode changes in distributional patterns of usage (By- sufficient information for distinguishing between bee, 2010). For instance, Perek(2016) investi- acceptable and unacceptable combinations. gates the productivity of the V the hell out of NP construction (e.g., You scared the hell out of me) 3 Distributional CxG Framework from 1930 to 2009. On one side, he clusters the We introduce a new framework aimed at integrat- vectors of verbs occurring in this construction to ing the computational representation derived from pin point the preferred semantic domains of the distributional methods into the explicit formaliza- Cx in
Load more

Recommended publications
  • Semantic Computing
    SEMANTIC COMPUTING Lecture 7: Introduction to Distributional and Distributed Semantics Dagmar Gromann International Center For Computational Logic TU Dresden, 30 November 2018 Overview • Distributional Semantics • Distributed Semantics – Word Embeddings Dagmar Gromann, 30 November 2018 Semantic Computing 2 Distributional Semantics Dagmar Gromann, 30 November 2018 Semantic Computing 3 Distributional Semantics Definition • meaning of a word is the set of contexts in which it occurs • no other information is used than the corpus-derived information about word distribution in contexts (co-occurrence information of words) • semantic similarity can be inferred from proximity in contexts • At the very core: Distributional Hypothesis Distributional Hypothesis “similarity of meaning correlates with similarity of distribution” - Harris Z. S. (1954) “Distributional structure". Word, Vol. 10, No. 2-3, pp. 146-162 meaning = use = distribution in context => semantic distance Dagmar Gromann, 30 November 2018 Semantic Computing 4 Remember Lecture 1? Types of Word Meaning • Encyclopaedic meaning: words provide access to a large inventory of structured knowledge (world knowledge) • Denotational meaning: reference of a word to object/concept or its “dictionary definition” (signifier <-> signified) • Connotative meaning: word meaning is understood by its cultural or emotional association (positive, negative, neutral conntation; e.g. “She’s a dragon” in Chinese and English) • Conceptual meaning: word meaning is associated with the mental concepts it gives access to (e.g. prototype theory) • Distributional meaning: “You shall know a word by the company it keeps” (J.R. Firth 1957: 11) John Rupert Firth (1957). "A synopsis of linguistic theory 1930-1955." In Special Volume of the Philological Society. Oxford: Oxford University Press. Dagmar Gromann, 30 November 2018 Semantic Computing 5 Distributional Hypothesis in Practice Study by McDonald and Ramscar (2001): • The man poured from a balack into a handleless cup.
    [Show full text]
  • A Comparison of Word Embeddings and N-Gram Models for Dbpedia Type and Invalid Entity Detection †
    information Article A Comparison of Word Embeddings and N-gram Models for DBpedia Type and Invalid Entity Detection † Hanqing Zhou *, Amal Zouaq and Diana Inkpen School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa ON K1N 6N5, Canada; [email protected] (A.Z.); [email protected] (D.I.) * Correspondence: [email protected]; Tel.: +1-613-562-5800 † This paper is an extended version of our conference paper: Hanqing Zhou, Amal Zouaq, and Diana Inkpen. DBpedia Entity Type Detection using Entity Embeddings and N-Gram Models. In Proceedings of the International Conference on Knowledge Engineering and Semantic Web (KESW 2017), Szczecin, Poland, 8–10 November 2017, pp. 309–322. Received: 6 November 2018; Accepted: 20 December 2018; Published: 25 December 2018 Abstract: This article presents and evaluates a method for the detection of DBpedia types and entities that can be used for knowledge base completion and maintenance. This method compares entity embeddings with traditional N-gram models coupled with clustering and classification. We tackle two challenges: (a) the detection of entity types, which can be used to detect invalid DBpedia types and assign DBpedia types for type-less entities; and (b) the detection of invalid entities in the resource description of a DBpedia entity. Our results show that entity embeddings outperform n-gram models for type and entity detection and can contribute to the improvement of DBpedia’s quality, maintenance, and evolution. Keywords: semantic web; DBpedia; entity embedding; n-grams; type identification; entity identification; data mining; machine learning 1. Introduction The Semantic Web is defined by Berners-Lee et al.
    [Show full text]
  • Distributional Semantics
    Distributional Semantics Computational Linguistics: Jordan Boyd-Graber University of Maryland SLIDES ADAPTED FROM YOAV GOLDBERG AND OMER LEVY Computational Linguistics: Jordan Boyd-Graber UMD Distributional Semantics 1 / 19 j j From Distributional to Distributed Semantics The new kid on the block Deep learning / neural networks “Distributed” word representations Feed text into neural-net. Get back “word embeddings”. Each word is represented as a low-dimensional vector. Vectors capture “semantics” word2vec (Mikolov et al) Computational Linguistics: Jordan Boyd-Graber UMD Distributional Semantics 2 / 19 j j From Distributional to Distributed Semantics This part of the talk word2vec as a black box a peek inside the black box relation between word-embeddings and the distributional representation tailoring word embeddings to your needs using word2vec Computational Linguistics: Jordan Boyd-Graber UMD Distributional Semantics 3 / 19 j j word2vec Computational Linguistics: Jordan Boyd-Graber UMD Distributional Semantics 4 / 19 j j word2vec Computational Linguistics: Jordan Boyd-Graber UMD Distributional Semantics 5 / 19 j j word2vec dog cat, dogs, dachshund, rabbit, puppy, poodle, rottweiler, mixed-breed, doberman, pig sheep cattle, goats, cows, chickens, sheeps, hogs, donkeys, herds, shorthorn, livestock november october, december, april, june, february, july, september, january, august, march jerusalem tiberias, jaffa, haifa, israel, palestine, nablus, damascus katamon, ramla, safed teva pfizer, schering-plough, novartis, astrazeneca, glaxosmithkline, sanofi-aventis, mylan, sanofi, genzyme, pharmacia Computational Linguistics: Jordan Boyd-Graber UMD Distributional Semantics 6 / 19 j j Working with Dense Vectors Word Similarity Similarity is calculated using cosine similarity: dog~ cat~ sim(dog~ ,cat~ ) = dog~ · cat~ jj jj jj jj For normalized vectors ( x = 1), this is equivalent to a dot product: jj jj sim(dog~ ,cat~ ) = dog~ cat~ · Normalize the vectors when loading them.
    [Show full text]
  • Sign-Based Construction Grammar: an Informal Synopsis IVA N A
    September 4, 2012 3 Sign-Based Construction Grammar: An Informal Synopsis IVA N A. SAG 1 Introduction This chapter1 is intended as an introduction to some of the central notions of Sign-Based Construction Grammar (SBCG).2 For a more general discussion of SBCG, including an informal, high-level summary of the framework, its historical development, motivation, and relation to other approaches to gram- mar, the reader is referred to Sag et al. this volume. 1.1 Preliminaries SBCG is a framework blending ideas developed over a quarter century of re- search in Head-Driven Phrase Structure Grammar (HPSG3) with those pre- 1Above all, I’d like to thank Paul Kay for hours of discussion and useful feedback at every stage in the seemingly endless process of writing this chapter. I’m also grateful to Chuck Fillmore and Laura Michaelis for many useful discussions of both content and exposition and for com- ments on an earlier draft, which was also been improved by the comments of Rui Chaves, Adele Goldberg, Bill Croft, Adam Przepiórkowski, Russell Lee-Goldman, and Hans Boas. In addition, discussions with the following people have been quite helpful: Farrell Ackerman, Emily Bender, Dan Flickinger, Mark Gawron, Andreas Kathol, Beth Levin, Bob Levine, Stefan Müller, Frank Van Eynde, Tom Wasow, Gert Webelhuth, and Steve Wechsler. None of the views expressed here should be attributed to any of these individuals; nor should they be held responsible for any remaining errors. 2SBCG owes a considerable debt to the implementation work carried out within CSLI’s LinGO Lab and the DELPHIN consortium, whose grammar development efforts have proceeded in paral- lel with the evolution of SBCG.
    [Show full text]
  • Facing the Facts of Fake: a Distributional Semantics and Corpus Annotation Approach Bert Cappelle, Pascal Denis, Mikaela Keller
    Facing the facts of fake: a distributional semantics and corpus annotation approach Bert Cappelle, Pascal Denis, Mikaela Keller To cite this version: Bert Cappelle, Pascal Denis, Mikaela Keller. Facing the facts of fake: a distributional semantics and corpus annotation approach. Yearbook of the German Cognitive Linguistics Association, De Gruyter, 2018, 6 (9-42). hal-01959609 HAL Id: hal-01959609 https://hal.archives-ouvertes.fr/hal-01959609 Submitted on 18 Dec 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Facing the facts of fake: a distributional semantics and corpus annotation approach Bert Cappelle, Pascal Denis and Mikaela Keller Université de Lille, Inria Lille Nord Europe Fake is often considered the textbook example of a so-called ‘privative’ adjective, one which, in other words, allows the proposition that ‘(a) fake x is not (an) x’. This study tests the hypothesis that the contexts of an adjective-noun combination are more different from the contexts of the noun when the adjective is such a ‘privative’ one than when it is an ordinary (subsective) one. We here use ‘embeddings’, that is, dense vector representations based on word co-occurrences in a large corpus, which in our study is the entire English Wikipedia as it was in 2013.
    [Show full text]
  • Sentiment, Stance and Applications of Distributional Semantics
    Sentiment, stance and applications of distributional semantics Maria Skeppstedt Sentiment, stance and applications of distributional semantics Sentiment analysis (opinion mining) • Aims at determining the attitude of the speaker/ writer * In a chunk of text (e.g., a document or sentence) * Towards a certain topic • Categories: * Positive, negative, (neutral) * More detailed Movie reviews • Excruciatingly unfunny and pitifully unromantic. • The picture emerges as a surprisingly anemic disappointment. • A sensitive, insightful and beautifully rendered film. • The spark of special anime magic here is unmistak- able and hard to resist. • Maybe you‘ll be lucky , and there‘ll be a power outage during your screening so you can get your money back. Methods for sentiment analysis • Train a machine learning classifier • Use lexicon and rules • (or combine the methods) Train a machine learning classifier Annotating text and training the model Excruciatingly Excruciatingly unfunny unfunny Applying the model pitifully unromantic. The model pitifully unromantic. Examples of machine learning Frameworks • Scikit-learn • NLTK (natural language toolkit) • CRF++ Methods for sentiment analysis • Train a machine learning classifier * See for instance Socher et al. (2013) that classified English movie review sentences into positive and negative sentiment. * Trained their classifier on 11,855 manually labelled sentences. (R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, C. Potts, Recursive deep models for semantic composition- ality over a sentiment treebank) Methods based on lexicons and rules • Lexicons for different polarities • Sometimes with different strength on the words * Does not need the large amount of training data that is typically required for machine learning methods * More flexible (e.g, to add new sentiment poles and new languages) Commercial use of sentiment analysis I want to retrieve everything positive and negative that is said about the products I sell.
    [Show full text]
  • Building Web-Interfaces for Vector Semantic Models with the Webvectors Toolkit
    Building Web-Interfaces for Vector Semantic Models with the WebVectors Toolkit Andrey Kutuzov Elizaveta Kuzmenko University of Oslo Higher School of Economics Oslo, Norway Moscow, Russia [email protected] eakuzmenko [email protected] Abstract approaches, which allowed to train distributional models with large amounts of raw linguistic data We present WebVectors, a toolkit that very fast. The most established word embedding facilitates using distributional semantic algorithms in the field are highly efficient Continu- models in everyday research. Our toolkit ous Skip-Gram and Continuous Bag-of-Words, im- has two main features: it allows to build plemented in the famous word2vec tool (Mikolov web interfaces to query models using a et al., 2013b; Baroni et al., 2014), and GloVe in- web browser, and it provides the API to troduced in (Pennington et al., 2014). query models automatically. Our system Word embeddings represent the meaning of is easy to use and can be tuned according words with dense real-valued vectors derived from to individual demands. This software can word co-occurrences in large text corpora. They be of use to those who need to work with can be of use in almost any linguistic task: named vector semantic models but do not want to entity recognition (Siencnik, 2015), sentiment develop their own interfaces, or to those analysis (Maas et al., 2011), machine translation who need to deliver their trained models to (Zou et al., 2013; Mikolov et al., 2013a), cor- a large audience. WebVectors features vi- pora comparison (Kutuzov and Kuzmenko, 2015), sualizations for various kinds of semantic word sense frequency estimation for lexicogra- queries.
    [Show full text]
  • Distributional Semantics for Understanding Spoken Meal Descriptions
    DISTRIBUTIONAL SEMANTICS FOR UNDERSTANDING SPOKEN MEAL DESCRIPTIONS Mandy Korpusik, Calvin Huang, Michael Price, and James Glass∗ MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA 02139, USA fkorpusik, calvinh, pricem, [email protected] ABSTRACT This paper presents ongoing language understanding experiments conducted as part of a larger effort to create a nutrition dialogue system that automatically extracts food concepts from a user’s spo- ken meal description. We first discuss the technical approaches to understanding, including three methods for incorporating word vec- tor features into conditional random field (CRF) models for seman- tic tagging, as well as classifiers for directly associating foods with properties. We report experiments on both text and spoken data from an in-domain speech recognizer. On text data, we show that the ad- dition of word vector features significantly improves performance, achieving an F1 test score of 90.8 for semantic tagging and 86.3 for food-property association. On speech, the best model achieves an F1 test score of 87.5 for semantic tagging and 86.0 for association. Fig. 1. A diagram of the flow of the nutrition system [4]. Finally, we conduct an end-to-end system evaluation through a user study with human ratings of 83% semantic tagging accuracy. Index Terms— CRF, Word vectors, Semantic tagging • We demonstrate a significant improvement in semantic tag- ging performance by adding word embedding features to a classifier. We explore features for both the raw vector values 1. INTRODUCTION and similarity to prototype words from each category. In ad- dition, improving semantic tagging performance benefits the For many patients with obesity, diet tracking is often time- subsequent task of associating foods with properties.
    [Show full text]
  • Constructions: a New Theoretical Approach to Language
    Review TRENDS in Cognitive Sciences Vol.7 No.5 May 2003 219 Constructions: a new theoretical approach to language Adele E. Goldberg Linguistics Department, University of Illinois, Urbana, IL 61801-0168, USA A new theoretical approach to language has emerged in the ‘core’ of language. Mainstream generative theory the past 10–15 years that allows linguistic observations argues further that the complexity of core language cannot about form–meaning pairings, known as ‘construc- be learned inductively by general cognitive mechanisms tions’, to be stated directly. Constructionist approaches and therefore learners must be hard-wired with principles aim to account for the full range of facts about that are specific to language (‘universal grammar’). language, without assuming that a particular subset of the data is part of a privileged ‘core’. Researchers in this Tenets of constructionist approaches field argue that unusual constructions shed light on Each basic tenet outlined below is shared by most more general issues, and can illuminate what is constructionist approaches. Each represents a major required for a complete account of language. divergence from the mainstream generative approach and, in many ways, a return to a more traditional view of Constructions – form and meaning pairings – have been language. the basis of major advances in the study of grammar since Tenet 1. All levels of description are understood to the days of Aristotle. Observations about specific linguistic involve pairings of form with semantic or discourse constructions have shaped our understanding of both function, including morphemes or words, idioms, particular languages and the nature of language itself. But partially lexically filled and fully abstract phrasal only recently has a new theoretical approach emerged that patterns.
    [Show full text]
  • Distributional Semantics Today Introduction to the Special Issue Cécile Fabre, Alessandro Lenci
    Distributional Semantics Today Introduction to the special issue Cécile Fabre, Alessandro Lenci To cite this version: Cécile Fabre, Alessandro Lenci. Distributional Semantics Today Introduction to the special issue. Revue TAL, ATALA (Association pour le Traitement Automatique des Langues), 2015, Sémantique distributionnelle, 56 (2), pp.7-20. hal-01259695 HAL Id: hal-01259695 https://hal.archives-ouvertes.fr/hal-01259695 Submitted on 26 Jan 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Distributional Semantics Today Introduction to the special issue Cécile Fabre* — Alessandro Lenci** * University of Toulouse, CLLE-ERSS ** University of Pisa, Computational Linguistics Laboratory ABSTRACT. This introduction to the special issue of the TAL journal on distributional seman- tics provides an overview of the current topics of this field and gives a brief summary of the contributions. RÉSUMÉ. Cette introduction au numéro spécial de la revue TAL consacré à la sémantique dis- tributionnelle propose un panorama des thèmes de recherche actuels dans ce champ et fournit un résumé succinct des contributions acceptées. KEYWORDS: Distributional Semantic Models, vector-space models, corpus-based semantics, se- mantic proximity, semantic relations, evaluation of distributional ressources. MOTS-CLÉS : Sémantique distributionnelle, modèles vectoriels, proximité sémantique, relations sémantiques, évaluation de ressources distributionnelles.
    [Show full text]
  • Distributional Semantics
    Distributional semantics Distributional semantics is a research area that devel- by populating the vectors with information on which text ops and studies theories and methods for quantifying regions the linguistic items occur in; paradigmatic sim- and categorizing semantic similarities between linguis- ilarities can be extracted by populating the vectors with tic items based on their distributional properties in large information on which other linguistic items the items co- samples of language data. The basic idea of distributional occur with. Note that the latter type of vectors can also semantics can be summed up in the so-called Distribu- be used to extract syntagmatic similarities by looking at tional hypothesis: linguistic items with similar distributions the individual vector components. have similar meanings. The basic idea of a correlation between distributional and semantic similarity can be operationalized in many dif- ferent ways. There is a rich variety of computational 1 Distributional hypothesis models implementing distributional semantics, includ- ing latent semantic analysis (LSA),[8] Hyperspace Ana- The distributional hypothesis in linguistics is derived logue to Language (HAL), syntax- or dependency-based from the semantic theory of language usage, i.e. words models,[9] random indexing, semantic folding[10] and var- that are used and occur in the same contexts tend to ious variants of the topic model. [1] purport similar meanings. The underlying idea that “a Distributional semantic models differ primarily with re- word is characterized by the company it keeps” was pop- spect to the following parameters: ularized by Firth.[2] The Distributional Hypothesis is the basis for statistical semantics. Although the Distribu- • tional Hypothesis originated in linguistics,[3] it is now re- Context type (text regions vs.
    [Show full text]
  • Introducing Sign-Based Construction Grammar IVA N A
    September 4, 2012 1 Introducing Sign-Based Construction Grammar IVA N A. SAG,HANS C. BOAS, AND PAUL KAY 1 Background Modern grammatical research,1 at least in the realms of morphosyntax, in- cludes a number of largely nonoverlapping communities that have surpris- ingly little to do with one another. One – the Universal Grammar (UG) camp – is mainly concerned with a particular view of human languages as instantia- tions of a single grammar that is fixed in its general shape. UG researchers put forth highly abstract hypotheses making use of a complex system of repre- sentations, operations, and constraints that are offered as a theory of the rich biological capacity that humans have for language.2 This community eschews writing explicit grammars of individual languages in favor of offering conjec- tures about the ‘parameters of variation’ that modulate the general grammat- ical scheme. These proposals are motivated by small data sets from a variety of languages. A second community, which we will refer to as the Typological (TYP) camp, is concerned with descriptive observations of individual languages, with particular concern for idiosyncrasies and complexities. Many TYP re- searchers eschew formal models (or leave their development to others), while others in this community refer to the theory they embrace as ‘Construction Grammar’ (CxG). 1For comments and valuable discussions, we are grateful to Bill Croft, Chuck Fillmore, Adele Goldberg, Stefan Müller, and Steve Wechsler. We also thank the people mentioned in footnote 8 below. 2The nature of these representations has changed considerably over the years. Seminal works include Chomsky 1965, 1973, 1977, 1981, and 1995.
    [Show full text]