Constructing grammar: A computational model of the emergence of early constructions Nancy Chih-Lin Chang Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2009-24 http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-24.html February 5, 2009 Copyright 2009, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Constructing grammar: A computational model of the emergence of early constructions by Nancy Chih-lin Chang A.B. (Harvard University) 1994 M.Phil. (Cambridge University) 1995 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor Jerome A. Feldman, Chair Professor Nelson Morgan Professor Daniel I. Slobin Fall 2008 The dissertation of Nancy Chih-lin Chang is approved. Chair Date Date Date University of California, Berkeley Fall 2008 Constructing grammar: A computational model of the emergence of early constructions Copyright c 2008 by Nancy Chih-lin Chang Abstract Constructing grammar: A computational model of the emergence of early constructions by Nancy Chih-lin Chang Doctor of Philosophy in Computer Science University of California, Berkeley Professor Jerome A. Feldman, Chair In this thesis I explore and formalize the view that grammar learning is driven by meaningful lan- guage use in context. On this view, the goal of a first language learner is to become a better language user — in particular, by acquiring linguistic constructions (structured mappings between form and meaning) that facilitate successful communication. I present a computational model in which all aspects of the language learning problem are reformulated in line with these assumptions. The representational basis of the model is a construction grammar formalism that captures constituent structure and relational constraints, both within and across the domains of form and meaning. This formalism plays a central role in two processes: language understanding, which uses constuctions to interpret utterances in context; and language learning, which seeks to improve comprehension by making judicious changes to the current set of constructions. The resulting integrated model of language structure, use and acquisition provides a cogni- tively motivated and computationally precise account of how children acquire their earliest multi- word constructions. I define a set of operations for proposing new constructions, either to capture contextually available mappings not predicted by the current grammar, or to reorganize existing constructions. Candidate constructions are evaluated using a minimum description length criterion that balances a structural bias toward simpler grammars against a data-driven bias toward more specific grammars. When trained with a corpus of child-directed utterances annotated with situ- ation descriptions, the model gradually acquires the concrete word combinations and item-based constructions that constitute the first steps toward adult language. Professor Jerome A. Feldman Dissertation Committee Chair 1 for my parents, I-Cheng and Phoebe Chang A word after a word after a word is power. — Margaret Atwood, from “Spelling” We are all talkers It is true, but underneath the talk lies The moving and not wanting to be moved, the loose Meaning, untidy and simple like a threshing floor. — John Ashbery, from “Soonest Mended” i Contents Dedication i Contents ii List of Figures v List of Tables vii Acknowledgments viii 1 Beyond single words 1 1.1 Meaningful language learning: the model in brief .................... 3 1.2 Goals in context ........................................ 6 1.2.1 Grammaticality and its discontents ......................... 7 1.2.2 Embodied models of language acquisition and use ............... 11 1.2.3 Contributions ..................................... 14 1.3 Road maps ........................................... 16 2 Orientations 18 2.1 Preliminaries .......................................... 19 2.1.1 Gold’s legacy ..................................... 19 2.1.2 Theoretical frameworks ............................... 22 2.1.3 The course of acquisition .............................. 26 2.2 Foundations .......................................... 32 2.2.1 Constructions ..................................... 32 2.2.2 Embodiment ..................................... 35 2.2.3 Usage .......................................... 39 2.3 Formalisms ........................................... 42 2.3.1 Unification-based grammar ............................. 43 2.3.2 Extending parsing to constructions ......................... 44 2.3.3 Embodiment and simulation ............................ 45 2.3.4 Learning as optimization .............................. 47 3 Embodied Construction Grammar 50 3.1 Overview ............................................ 51 3.1.1 From sentence to simulation ............................ 53 3.1.2 Running example in ECG .............................. 55 3.1.3 Beyond Harry ran home ................................ 59 3.1.4 The formalism at a glance .............................. 61 3.2 Schemas ............................................ 64 3.2.1 Form .......................................... 64 3.2.2 Meaning ........................................ 65 3.2.3 Function ........................................ 68 3.3 Constructions ......................................... 69 ii 3.3.1 Cross-domain mapping types ............................ 70 3.3.2 Reference and predication .............................. 71 3.3.3 Other constructions .................................. 76 3.4 Summary ............................................ 79 4 Using embodied constructions 80 4.1 Definitions ........................................... 82 4.1.1 Schemas and constructions ............................. 82 4.1.2 Utterances and contexts ............................... 86 4.1.3 Analysis and resolution ............................... 88 4.2 A simple analyzer-resolver .................................. 91 4.2.1 Constructional analysis ............................... 93 4.2.2 Contextual resolution ................................ 95 4.3 Evaluation criteria ....................................... 99 4.3.1 Scoring construct graphs ............................... 100 4.3.2 Scoring form and meaning coverage ........................ 102 4.3.3 Scoring (resolved) analyses ............................. 104 5 Learning embodied constructions 111 5.1 The child’s learning task: a review ............................. 111 5.1.1 Representational requirements ........................... 112 5.1.2 Comprehension-based constraints ......................... 113 5.1.3 Developmental desiderata .............................. 113 5.2 Problem statement(s) ..................................... 115 5.2.1 Construction learning defined ........................... 115 5.2.2 Hypothesis space ................................... 117 5.2.3 Prior knowledge ................................... 118 5.2.4 Input data ....................................... 120 5.2.5 Performance criteria ................................. 121 5.2.6 Summary: relational construction learning .................... 122 5.3 Approaches to learning .................................... 123 5.3.1 Bayesian inference .................................. 124 5.3.2 Bayesian model merging ............................... 125 5.3.3 Model merging for embodied constructions .................... 128 5.4 Usage-based construction learning: overview ....................... 130 6 Usage-based learning operations 134 6.1 Overview of operations ................................... 135 6.2 Context-driven mapping operations ............................ 136 6.2.1 Simple mapping ................................... 137 6.2.2 Relational mapping .................................. 138 6.3 Reorganization operations .................................. 143 6.3.1 Structural alignment ................................. 144 6.3.2 Merging ........................................ 148 6.3.3 Joining ......................................... 156 6.3.4 Splitting ........................................ 158 6.4 Summary ............................................ 160 7 Simplicity-based evaluation heuristics 163 7.1 Minimum description length ................................ 163 7.2 MDL for ECG ......................................... 165 7.2.1 Counts and weights ................................. 167 7.2.2 Grammar length ................................... 168 7.2.3 Data length ...................................... 169 7.3 Updating description length ................................. 172 iii 7.3.1 Updating grammar length .............................. 172 7.3.2 Updating data length ................................. 174 7.4 Incorporating reference costs ................................ 175 7.5 Discussion ........................................... 177 8 The model in action 179 8.1 Nomi’s world ......................................... 179 8.1.1 Input assumptions .................................. 180 8.1.2 Initial lexicon and schema set ...........................