Distributional Semantics Meets Construction Grammar. Towards a Unified Usage-Based Model of Grammar and Meaning Giulia Rambelli Emmanuele Chersoni University of Pisa The Hong Kong Polytechnic University [email protected]@gmail.com Philippe Blache Chu-Ren Huang Aix-Marseille University The Hong Kong Polytechnic University [email protected] [email protected] Alessandro Lenci University of Pisa [email protected] Abstract Obj1 Obj2]). It is worth stressing that, even if the concept of construction is based on the idea In this paper, we propose a new type of seman- that linguistic properties actually emerge from lan- tic representation of Construction Grammar guage use, CxG theories have typically preferred that combines constructions with the vector to model the semantic content of constructions in representations used in Distributional Seman- tics. We introduce a new framework, Distribu- terms of hand-made, formal representations like tional Construction Grammar, where grammar those of Frame Semantics (Baker et al., 1998). and meaning are systematically modeled from This leaves open the issue of how semantic repre- language use, and finally, we discuss the kind sentations can be learned from empirical evidence, of contributions that distributional models can and how do they relate to the usage-based nature of provide to CxG representation from a linguis- Cxs. In fact, for a usage-based model of grammar tic and cognitive perspective. based on a strong syntax-semantics parallelism, it would be desirable to be grounded on a framework 1 Introduction allowing to learn the semantic content of Cxs from In the last decades, usage-based models of lan- language use. guage have captured the attention of linguistics In this perspective, a promising solution for and cognitive science (Tommasello, 2003; Bybee, representing constructional semantics is given by 2010). The different approaches covered by this an approach to meaning representations that has label are based on the assumptions that linguistic gained a rising interest in both computational lin- knowledge is embodied in mental processing and guistics and cognitive science, namely Distribu- representations that are sensitive to context and tional Semantics (henceforth DS). DS is a usage- statistical probabilities (Boyland, 2009), and that based model of word meaning, based on the well- language structures at all levels, from morphology established assumption that the statistical distribu- to syntax, emerge out of facts of actual language tion of linguistic items in context plays a key role usage (Bybee, 2010). in characterizing their semantic behaviour (Distri- A usage-based framework that turned out to be butional Hypothesis (Harris, 1954)). More pre- extremely influential is Construction Grammar cisely, Distributional Semantic Models (DSMs) (CxG) (Hoffman and Trousdale, 2013), a family represent the lexicon in terms of vector spaces, of theories sharing the fundamental idea that lan- where a lexical target is described in terms of a guage is a collection of form-meaning pairings vector (also known as embedding) built by identi- called constructions (henceforth Cxs)(Fillmore, fying in a corpus its syntactic and lexical contexts 1988; Goldberg, 2006). Cxs differ for their degree (Lenci, 2018). Lately, neural models to learn dis- of schematicity, ranging from morphemes (e.g., tributional vectors have gained massive popular- pre-, -ing), to complex words (e.g., daredevil) to ity: these algorithms build low-dimensional vec- filled or partially-filled idioms (e.g., give the devil tor representations by learning to optimally predict his dues or Jog (someones) memory) to more ab- the contexts of the target words (Mikolov et al., stract patterns like the ditransitive Cxs [Subj V 2013). On the negative side, DS lacks a clear 110 Proceedings of the First International Workshop on Designing Meaning Representations, pages 110–120 Florence, Italy, August 1st, 2019 c 2019 Association for Computational Linguistics connection with usage-based theoretical frame- has never formulated a systematic proposal for de- works. To the best of our knowledge, existing riving representations of constructional meaning attempts of linking DS with models of grammar from corpus data. Previous literature has mostly have rather targeted formal theories like Montague focused either on the automatic identification of Grammar and Categorial Grammar (Baroni et al., constructions on the basis of their formal features, 2014; Grefenstette and Sadrzadeh, 2015). or on modeling the meaning of a specific CxG. To sum up, both CxG and DS share the assump- For the former approach, we should mention tion that linguistic structures naturally emerge the works of Dunn(2017, 2019) that aim at au- from language usage, and that a representation tomatically inducing a set of grammatical units of both form and meaning of any linguistic item (Cxs) from a large corpus. On the one hand, can be modeled through its distributional statistics, Dunn’s contributions provide a method for extract- and more generally, with the quantitative informa- ing Cxs from corpora, but on the other hand they tion derived from corpus data. However, these two are mainly concerned with the formal side of the models still live in parallel worlds. On the one constructions, and especially with the problem of hand, CxG is a model of grammar in search for how syntactic constraints are learned. Some sort a consistent usage-based model of meaning, and, of semantic representation is included, in the form conversely, DS is a computational framework to of semantic cluster of word embeddings to which build semantic representations in search for an em- the word forms appearing in the constructions are pirically adequate theory of grammar. assigned. However, these works do not present As we illustrate in Section 2, occasional en- any evaluation of the construction representations counters between DS and CxG have already hap- in terms of semantic tasks. pened, but we believe that new fruitful advances Another line of research has focused in using could come from the exploitation of the mu- constructions for building computational models tual synergies between CxG and DS, and by let- of language acquisition. Alishahi and Stevenson ting these two worlds finally meet and interact (2008) propose a model for the representation, ac- in a more systematic way. Following this direc- quisition and use of verb argument structure by tion of research, we introduce a new representa- formulating constructions as probabilistic associ- tion framework called Distributional Construc- ations between syntactic and semantic properties tion Grammar, which aims at bringing together of verbs and their arguments. This probabilistic these two theoretical paradigms. Our goal is to association emerges over time through a Bayesian integrate distributional information into construc- acquisition process in which similar verb usages tions by completing their semantic structures with are detected and grouped together to form general distributional vectors extracted from large textual constructions, based on their syntactic and seman- corpora, as samples of language usage. tic properties. Despite the success of this model, These pages are structured as follows: after the semantic representation of argument structure reviewing existing literature on CxG and related is still symbolic and each semantic category of in- computational studies, in Section3 we outline put constructions are manually compiled, in con- the key characteristics of our theoretical proposal, trast with the usage-based nature of constructions. while Section4 provides a general discussion Other studies used DSMs to model construc- about what contributions DSMs can provide to tional meaning, by focusing on a specific type of CxG representation from a linguistic and cognitive Cx rather than on the entire grammar. For exam- perspective. Although this is essentially a theoret- ple, Levshina and Heylen(2014) build a vector ical contribution, we outline ongoing work focus- space to study Dutch causative constructions with ing on its computational implementation and em- doen (‘do’) and laten (‘let’). They compute several pirical validation. We conclude by reporting future vector spaces with different context types, both for perspectives of research. the nouns that fill the Causer and Causee slot and for the verbs that fill the Effected Predicate slot. 2 Related Work Then, they cluster these nouns and verbs at differ- Despite the popularity of the constructional ap- ent levels of granularity and test which classifica- proach in corpus linguistics (Gries and Stefanow- tion better predicts the use of laten and doen. itsch, 2004), computational semantics research A recent trend in diachronic linguistics investi- 111 gates linguistic change as a sequence of gradual for some alternations, verb embeddings encode changes in distributional patterns of usage (By- sufficient information for distinguishing between bee, 2010). For instance, Perek(2016) investi- acceptable and unacceptable combinations. gates the productivity of the V the hell out of NP construction (e.g., You scared the hell out of me) 3 Distributional CxG Framework from 1930 to 2009. On one side, he clusters the We introduce a new framework aimed at integrat- vectors of verbs occurring in this construction to ing the computational representation derived from pin point the preferred semantic domains of the distributional methods into the explicit formaliza- Cx in