Knowledge and Learning in Natural Language BARKER by MASSACHUSET TS INSTITUTE OF TECHNOLOGY Charles D. Yang A PR 2 4 2001 B.S., Case Western Reserve University (1994) LIBRARIES S.M., Massachusetts Institute of Technology (1997) Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY August 2000 @ Massachusetts Institute of Technology 2000 All rights reserved Author- Department of Ictrical Eng eering and Computer Science August 2000 Certified by Robert C. Berwick Thesis Supervisor Certified by Noam Chomsky "The4i Supervisor Accepted by Arthur C. Smith Chairman, Committee on Graduate Students Knowledge and Learning in Natural Language by Charles D. Yang Submitted to the Department of Electrical Engineering and Computer Science August 2000 In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Abstract The present dissertation is a study of language development in children. From a biological perspective, the development of language, as the development of any other organic systems, is an interaction between internal and external factors; specifically, between the child's internal knowledge of linguistic structures and the external linguistic experience he receives. Drawing insights from the study of biological evolution, we put forth a quantitative model of language acquisition that make this interaction precise, by embedding a theory of knowledge, the Universal Grammar, into a theory of learning from experience. In particular, we advance the idea that language acquisition should be modeled as a population of grammatical hypotheses, competing to match the external linguistic experiences, much like in a natural selection process. We present evidence - conceptual, mathematical, and empirical, and from a number of independent areas of linguistic research, including the acquisition of syntax and morphophology, and historical language change - to demonstrate the model's correctness and utility. Thesis Supervisor: Robert C. Berwick Title: Professor of Computer Science and Brain and Cognitive Science Thesis Supervisor: Noam Chomsky Title: Institute Professor of Linguistics and Philosophy To my parents and gramma. "My sons, dig in the vineyard," were the last words of the old man in the fable: and, though the sons found no treasure, they made their fortune by the grapes. - T. H. Huxley Acknowledgments The heresies advocated in these pages, if of any value at all, result from digging for treasures in a vineyard but harvesting grapes instead. Or, in less romantic terms, the present thesis started out a pure accident. In the spring of 1998, I took an excellent course on evolutionary biology with two of the best evolutionary biologists around: Dick Lewontin and Stephen Jay Gould. There were plenty of treasures but I managed to find some grapes as well. On a field trip to the American Museum of Natural History, while waiting impatiently for SJG for a guided tour, I started on the class reading material, Dick Lewontin's 1983 paper Organism as Subject and Object of Evolution, which points out a conceptual feature of evolutionary process that cannot be exaggerated. Evolutionary change is a variational process. That is, it is not that individuals undergo direct change themselves; rather, it is the distribution of different individuals that changes under evolutionary forces. At that very moment, it occurred to me that language acquisition might be understood in a similar fashion: the distribution of Universal Grammar adaptively changes to the linguistic evidence presented to the child learner. The thesis explores the consequences of this idea. Of course, knowing where to dig was only a start. In cultivating grapes I have re- ceived enormous help from my teachers and committee members. Noam Chomsky and Bob Berwick, my co-supervisors, are jointly responsible for every step in and every aspect of my intellectual development such that words of gratitude are hardly needed for they could not be adequate. Many thanks to Tom Roeper, who kindly took me under his wings and whose energy is only matched by his immense knowledge of child language. Many thanks also to Patrick Winston, for his good scientific sense and much needed support and advice during the final stage of my graduate school. The list of supervisors does not stop with the committee: the unofficial members of the committee have been equally instrumental, and I thank them all wholeheartedly. Comrades John Frampton and Sam Gutmann, fellow members of Team Rocket Science, insisted on getting the math right and always made sure my thinking is without inconsistencies; I look forward to many more meetings at Toscanini's. Morris Halle taught me everything I know about phonology and devoted an embarrassing amount of time to this work, particularly the chapter on irregular verbs, which is dedicated to him. And, of course, Julie Legate, my best friend and critic and also member of the Team, who read everything I've ever written and contributed more than anyone else to this thesis in numerous ways. In addition, I am indebted to many teachers, colleagues, and friends, too numerous to list here, for suggestions and comments on this thesis and other projects, and for being an im- portant part of my graduate education. I especially want to thank Eric Grimson for clearing many hurdles during my time at MIT. The financial support for my education came from a National Science Foundation fellowship, supplemented by various grants from Bob Berwick, no string attached. Gratitude also goes to the NEC Research Institute, Sun Microsystems Research Labs, and Microsoft Research Institute, where I held summer internships, which not only provided additional income but also helped shape my perspectives on many things. The boring life of a graduate student has been made less so by friends in the MIT AI Lab, a unique and remarkable place where free thinking, free research, free lunch, and free fun are institutionally encouraged and pursued. I wish graduate students in all disciplines at all places could be equally free. Thanks to Andrew Berry for tips on how not to make it in academia, Jeff Brennan for coffee and trash talk, Adriane Desmond for two superb intel- lectual biographies, Dave Evans for soccer and beer, Shigeru Miyamoto for Zelda, Carolyn Smallwood for scrabble and Jack, and the past residents of the Church Street Inn for all the above. Special thanks to Frank Wolff, who gave me a copy of Language and Mind when I was a college senior. Julie has proved that doing linguistics was the best decision I've ever made: this thesis certainly would not be possible without her, but neither would much much more. In fact, life without her is unthinkable. Russell has given new meaning to what this is all about: I only wish he knew it as well. I thank our families for their support and encouragement. My parents and my gramma are the best teachers I've ever had: to them I dedicate this thesis. Now, on to the vineyard. Contents 1 The Study of Language and Language Acquisition 10 1.1 The Naturalistic Approach to Language . ...... ....... ....... 10 1.2 The Structure of Language Acquisition ....... ......... ...... 13 1.2.1 Form al Sufficiency ........ ......... ........ .... 14 1.2.2 Developmental Compatibility ..... ....... ...... ..... 15 1.2.3 Explanatory Continuity ......... ............. .... 17 1.3 A R oadm ap .......... .............. ............. 19 2 A Variational Model of Language Acquisition 21 2.1 Against Transformational Learning .. ......... ......... .... 22 2.1.1 Formal Insufficiency of the Triggering Model ........ ....... 24 2.1.2 Developmental Incompatibility of the Triggering Model ....... .. 26 2.1.3 Imperfection in Child Language? ....... ........ ...... 28 2.2 The Variational Approach to Language Acquisition .... ....... .... 30 2.2.1 The Dynamics of Darwinian Evolution ......... ........ 30 2.2.2 Language Acquisition as Grammar Competition ...... ....... 31 2.3 The Dynamics of Variational Learning ... ....... ....... ..... 35 2.3.1 Asymptotic Behaviors ....... ...... ...... ....... 35 2.3.2 Stable Multiple Grammars .. ......... ........ ...... 37 2.3.3 Unambiguous evidence ... ...... ....... ...... ..... 39 2.4 Learning Grammars in a Parametric Space ............ ........ 41 2.4.1 Parameters Make Learning Efficient ............ ....... 41 7 2.4.2 Parameter Expression and Cues .. ..... ..... .... ..... 44 2.5 Related Approaches .. ....... ...... ...... ...... .... 45 3 Competing Grammars in Child Syntax 48 3.1 The Time Courses of Three Parameters . .......... 49 3.1.1 Verb Raising and Subject Drop: the Baselines . ... .. 50 3.1.2 V1 Patterns in V2 Learners .............. .. 51 3.2 A Quantitative Argument from the Poverty of Stimulus . .. 54 3.3 The Nature of Null Subjects in Children ........... .. 57 3.3.1 The Early Acquisition of Chinese and Italian Subject Drop . .. 59 3.3.2 Why English Kids (Sometimes) Use Chinese ..... 61 3.4 Sum m ary ............................ 63 4 Words, Rules, and Competitions 65 4.1 Background ....... ....... ........ ....... .. .. .. 65 4.2 A Model of Rule Competition .................... .. .. 68 4.2.1 A Simple Learning Task ................... .. .. 69 4.2.2 R ules ............................. .. .. 70 4.2.3 Rule Competition ....................... .. .. 73 4.2.4 The Absolute and Stochastic Blocking Principles ..... .. .. 78 4.3 Words vs. Rules in Overregularization .... ........... .. ... 79 4.3.1