Formal Language Theory: Refining the Chomsky Hierarchy
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
The Mathematics of Syntactic Structure: Trees and Their Logics
Computational Linguistics Volume 26, Number 2 The Mathematics of Syntactic Structure: Trees and their Logics Hans-Peter Kolb and Uwe MÈonnich (editors) (Eberhard-Karls-UniversitÈat Tubingen) È Berlin: Mouton de Gruyter (Studies in Generative Grammar, edited by Jan Koster and Henk van Riemsdijk, volume 44), 1999, 347 pp; hardbound, ISBN 3-11-016273-3, $127.75, DM 198.00 Reviewed by Gerald Penn Bell Laboratories, Lucent Technologies Regular languages correspond exactly to those languages that can be recognized by a ®nite-state automaton. Add a stack to that automaton, and one obtains the context- free languages, and so on. Probably all of us learned at some point in our univer- sity studies about the Chomsky hierarchy of formal languages and the duality be- tween the form of their rewriting systems and the automata or computational re- sources necessary to recognize them. What is perhaps less well known is that yet another way of characterizing formal languages is provided by mathematical logic, namely in terms of the kind and number of variables, quanti®ers, and operators that a logical language requires in order to de®ne another formal language. This mode of characterization, which is subsumed by an area of research called descrip- tive complexity theory, is at once more declarative than an automaton or rewriting system, more ¯exible in terms of the primitive relations or concepts that it can pro- vide resort to, and less wedded to the tacit, not at all unproblematic, assumption that the right way to view any language, including natural language, is as a set of strings. -
Philosophical Problems in Child Language Research the Naive
Philosophical Problems in Child Language Research The Naive Realism of Traditional Semantics John Lyons, Oxford linguist, writes in his book (1995) on linguistic semantics: “in my view, it is impossible to evaluate even the most down-to-earth and apparently unphilosophical works in descriptive semantics unless one has some notion of the general philosophical framework within which they are written. This holds true regardless of whether the authors themselves are aware of the philosophical origins or implications of their working principles” (p. 88). One can only agree with Lyons’ position. And one can only admire his honesty when, later, he acknowledges: “throughout this book I have adopted the viewpoint of naive realism, according to which the ontological structure of the world is objectively independent both of perception and cognition and also of language. As we have dealt with particular topics, this view has been gradually elaborated (and to some degree modified); and a more technical metalanguage has been developed in order to formulate with greater precision than is possible in the everyday metalanguage the relations of reference and denotation that hold between language and the world. According to the viewpoint adopted here, the world contains a number of first-order entities (with first-order properties) which fall into certain ontological categories (or natural kinds); it also contains aggregates of stuff or matter(with first-order properties), portions of which can be individuated, quantified, enumerated—and thus treated linguistically as entities—by using the lexical and grammatical resources of particular natural languages. All natural languages, it may be assumed, provide their users with the means of referring to first-order entities and expressing propositions which describe them in terms of their (first-order) properties, actual or ascribed, essential or contingent: such languages have the expressive power of first-order formal languages” (p. -
Hierarchy and Interpretability in Neural Models of Language Processing ILLC Dissertation Series DS-2020-06
Hierarchy and interpretability in neural models of language processing ILLC Dissertation Series DS-2020-06 For further information about ILLC-publications, please contact Institute for Logic, Language and Computation Universiteit van Amsterdam Science Park 107 1098 XG Amsterdam phone: +31-20-525 6051 e-mail: [email protected] homepage: http://www.illc.uva.nl/ The investigations were supported by the Netherlands Organization for Scientific Research (NWO), through a Gravitation Grant 024.001.006 to the Language in Interaction Consortium. Copyright © 2019 by Dieuwke Hupkes Publisher: Boekengilde Printed and bound by printenbind.nl ISBN: 90{6402{222-1 Hierarchy and interpretability in neural models of language processing Academisch Proefschrift ter verkrijging van de graad van doctor aan de Universiteit van Amsterdam op gezag van de Rector Magnificus prof. dr. ir. K.I.J. Maex ten overstaan van een door het College voor Promoties ingestelde commissie, in het openbaar te verdedigen op woensdag 17 juni 2020, te 13 uur door Dieuwke Hupkes geboren te Wageningen Promotiecommisie Promotores: Dr. W.H. Zuidema Universiteit van Amsterdam Prof. Dr. L.W.M. Bod Universiteit van Amsterdam Overige leden: Dr. A. Bisazza Rijksuniversiteit Groningen Dr. R. Fern´andezRovira Universiteit van Amsterdam Prof. Dr. M. van Lambalgen Universiteit van Amsterdam Prof. Dr. P. Monaghan Lancaster University Prof. Dr. K. Sima'an Universiteit van Amsterdam Faculteit der Natuurwetenschappen, Wiskunde en Informatica to my parents Aukje and Michiel v Contents Acknowledgments xiii 1 Introduction 1 1.1 My original plan . .1 1.2 Neural networks as explanatory models . .2 1.2.1 Architectural similarity . .3 1.2.2 Behavioural similarity . -
The Complexity of Narrow Syntax : Minimalism, Representational
Erschienen in: Measuring Grammatical Complexity / Frederick J. Newmeyer ... (Hrsg.). - Oxford : Oxford University Press, 2014. - S. 128-147. - ISBN 978-0-19-968530-1 The complexity of narrow syntax: Minimalism, representational economy, and simplest Merge ANDREAS TROTZKE AND JAN-WOUTER ZWART 7.1 Introduction The issue of linguistic complexity has recently received much attention by linguists working within typological-functional frameworks (e.g. Miestamo, Sinnemäki, and Karlsson 2008; Sampson, Gil, and Trudgill 2009). In formal linguistics, the most prominent measure of linguistic complexity is the Chomsky hierarchy of formal languages (Chomsky 1956), including the distinction between a finite-state grammar (FSG) and more complicated types of phrase-structure grammar (PSG). This dis- tinction has played a crucial role in the recent biolinguistic literature on recursive complexity (Sauerland and Trotzke 2011). In this chapter, we consider the question of formal complexity measurement within linguistic minimalism (cf. also Biberauer et al., this volume, chapter 6; Progovac, this volume, chapter 5) and argue that our minimalist approach to complexity of derivations and representations shows simi- larities with that of alternative theoretical perspectives represented in this volume (Culicover, this volume, chapter 8; Jackendoff and Wittenberg, this volume, chapter 4). In particular, we agree that information structure properties should not be encoded in narrow syntax as features triggering movement, suggesting that the relevant information is established at the interfaces. Also, we argue for a minimalist model of grammar in which complexity arises out of the cyclic interaction of subderivations, a model we take to be compatible with Construction Grammar approaches. We claim that this model allows one to revisit the question of the formal complexity of a generative grammar, rephrasing it such that a different answer to the question of formal complexity is forthcoming depending on whether we consider the grammar as a whole, or just narrow syntax. -
What Was Wrong with the Chomsky Hierarchy?
What Was Wrong with the Chomsky Hierarchy? Makoto Kanazawa Hosei University Chomsky Hierarchy of Formal Languages recursively enumerable context- sensitive context- free regular Enduring impact of Chomsky’s work on theoretical computer science. 15 pages devoted to the Chomsky hierarchy. Hopcroft and Ullman 1979 No mention of the Chomsky hierarchy. 450 INDEX The same with the latest edition of Carmichael, R. D., 444 CNF-formula, 302 Cartesian product, 6, 46 Co-Turing-recognizableHopcroft, language, 209 Motwani, and Ullman (2006). CD-ROM, 349 Cobham, Alan, 444 Certificate, 293 Coefficient, 183 CFG, see Context-free grammar Coin-flip step, 396 CFL, see Context-free language Complement operation, 4 Chaitin, Gregory J., 264 Completed rule, 140 Chandra, Ashok, 444 Complexity class Characteristic sequence, 206 ASPACE(f(n)),410 Checkers, game of, 348 ATIME(t(n)),410 Chernoff bound, 398 BPP,397 Chess, game of, 348 coNL,354 Chinese remainder theorem, 401 coNP,297 Chomsky normal form, 108–111, 158, EXPSPACE,368 198, 291 EXPTIME,336 Chomsky, Noam, 444 IP,417 Church, Alonzo, 3, 183, 255 L,349 Church–Turing thesis, 183–184, 281 NC,430 CIRCUIT-SAT,386 NL,349 Circuit-satisfiability problem, 386 NP,292–298 CIRCUIT-VALUE,432 NPSPACE,336 Circular definition, 65 NSPACE(f(n)),332 Clause, 302 NTIME(f(n)),295 Clique, 28, 296 P,284–291,297–298 CLIQUE,296 PH,414 Sipser 2013Closed under, 45 PSPACE,336 Closure under complementation RP,403 context-free languages, non-, 154 SPACE(f(n)),332 deterministic context-free TIME(f(n)),279 languages, 133 ZPP,439 P,322 Complexity -
7. Pragmatics
Oxford Handbook of Computational Linguistics, 2nd edn. Drafted Apr 20, 2011; revised Mar 4, 2014 7. Pragmatics Christopher Potts Department of Linguistics, Stanford University Abstract This chapter reviews core empirical phenomena and technical concepts from linguistic pragmatics, including context dependence, common ground, indexicality, conversational implicature, presupposition, and speech acts. I seek to identify unifying themes among these concepts, provide a high-level guide to the primary literature, and relate the phenomena to issues in computational linguistics. Keywords context dependence, common ground, indexicality, conversational implica- ture, presupposition, speech acts 7.1 Introduction The linguistic objects that speakers use in their utterances vastly underdetermine the contributions that those utterances make to discourse. To borrow an analogy from Levinson(2000: 4), “an utterance is not [... ] a veridical model or ‘snapshot’ of the scene it describes”. Rather, the encoded content merely sketches what speakers intend and hearers perceive. The fundamental questions of pragmatics concern how semantic content, contextual information, and general communicative pressures interact to guide discourse participants in filling in these sketches: (i) How do language users represent the context of utterance, and which aspects of the context are central to communication? (ii) We often “mean more than we say”. What principles guide this pragmatic enrichment, and how do semantic values (conventional aspects of meaning) enable and constrain -
Unit 3 Looking at Data-2
UNIT 3 LOOKING AT DATA-2 Structure Objectives Formal Linguistics - An Introduction Generative Grammar 3.2.1 Principal Goals Generativists and Structuralists 3.3.1 Generativists and Bloomfieldians Tranformational Generative Grammar Let Us Sum Up Key Words Questions Suggested Readings 3.0 OBJECTIVES , In this unit, we will discuss o the Generative framework of grammar o its differences with the Structuralists We will also briefly talk about Transformational Generative Grammar. Some of these ideas may appear difficult but they will be clear to you as you read the course. Don't get discouraged if some concepts appear difficult and complex. They will be clarified as we proceed along. 3.1 FORMAL LINGLTISTICS - AN INTRODUCTION In the early 1950fs,signs of restiveness began to disturb the calm of structuralism, and by the end of the decahe new ideas emerged in a big way. Chomsky, a student of Zellig Harris was concerned with discovering a general theory of grammatical structure. He believed that an adequate grammar should provide a basis for explaining how sentences are used and understood. He reproaches the Bloonlfieldeans for "their satisfaction with description [and] their refusal to explain" ( 198 1:38). According to him, as other 'developing sciences, linguistics should also endeavour to establish a more ambitious goal than mere description and classification. Linguists should aim at developing methods not just for the description of language but also for understanding the nature of language. And this was possible only if one takes recourse to intuition of native speakers. Intuition had, however, remained a source of discomfiture for Bloomfield. -
14 the Linguistic Structure of Discourse
The Linguistic Structure of Discourse 265 14 The Linguistic Structure of Discourse LIVIA POLANYI 0 Introduction We take as the goal of our formal work in discourse analysis explaining how speakers almost without error interpret personal, temporal, and spatial deixis, recover the objects of anaphoric mention, and produce responses which demonstrate that they know what is going on in the talk despite disturbances to orderly discourse develop- ment (Polanyi 1978, 1987, 1996; Polanyi and van den Berg 1996; Polanyi and Scha 1984; Scha and Polanyi 1988). With the informal description of the Linguistic Discourse Model (LDM) in this chapter, we take a first step toward this goal by proposing answers to three basic questions: what are the atomic units of discourse? What kinds of structures can be built from the elementary units? How are the resulting struc- tures interpreted semantically? After sketching machinery to account for discourse segmentation, parsing, and interpretation, we will conclude by addressing the con- cerns readers of the present volume may have about the utility of a formal theory of discourse. What sort of work could such a theory do? To argue for our approach, we will examine the data and discussion presented in Prince’s (1988) account of Yiddish expletive, ES + Subject Postposing. We will use the Yiddish data to show how the LDM analysis allows us to give a much more principled account than has been possible before of what it means for an entity to be “new in the discourse.” In the concluding section of the chapter, we will broaden the discussion to argue for the benefits to sociolinguists and other discourse researchers of the formal methods we have proposed. -
Generative Linguistics Within the Cognitive Neuroscience of Language Alec Marantz
Generative linguistics within the cognitive neuroscience of language Alec Marantz Department of Linguistics and Philosophy, MIT Standard practice in linguistics often obscures the connection between theory and data, leading some to the conclusion that generative linguistics could not serve as the basis for a cognitive neuroscience of language. Here the foundations and methodology of generative grammar are clarified with the goal of explaining how generative theory already functions as a reasonable source of hypotheses about the representation and computation of language in the mind and brain. The claims of generative theory, as exemplified, e.g., within Chomsky’s (2000) Minimalist Program, are contrasted with those of theories endorsing parallel architectures with independent systems of generative phonology, syntax and semantics. The single generative engine within Minimalist approaches rejects dual routes to linguistic representations, including possible extra-syntactic strategies for semantic structure-building. Clarification of the implications of this property of generative theory undermines the foundations of an autonomous psycholinguistics, as established in the 1970’s, and brings linguistic theory back to the center of a unified cognitive neuroscience of language. 2 1. The place of linguistics* The first decade of the 21st century should be a golden era for the cognitive neuroscience of language. Fifty years of contemporary linguistic analysis of language can be coupled with a wide range of brain imaging and brain monitoring machines to test hypotheses and refine theory and understanding. However, there is still a gulf between mainstream linguistics within the generative linguistic tradition and most of those engaged in experimental cognitive neuroscience research. Some have argued that the fault here lies with the linguists, whose generative theories are based in principle on separating the study of linguistic representations from research on the acquisition and use of language in the minds and brains of speakers. -
Structural Linguistics and Formal Semantics
Structural Linguistics And Formal Semantics Jaroslav Peregrin [Travaux du Cercle Linguistique de Prague I, Benjamins, Amsterdam, 1995; original pagination] Introduction The beginning of this century hailed a new paradigm in linguistics, the paradigm brought about by de Saussure's Cours de Linguistique Genérále and subsequently elaborated by Jakobson, Hjelmslev and other linguists. It seemed that the linguistics of this century was destined to be structuralistic. However, half of the century later a brand new paradigm was introduced by Chomsky's Syntactic Structures followed by Montague's formalization of semantics. This new turn has brought linguistics surprisingly close to mathematics and logic, and has facilitated a direct practical exploitation of linguistic theory by computer science. One of the claims of this paper is that the post-Saussurian structuralism, both in linguistics and in philosophy, is partly based on ideas quite alien to de Saussure. The main aim then is to explain the ideas driving the formalistic turn of linguistics and to investigate the problem of the extent to which they can be accommodated within the framework of the Saussurian paradigm. The main thesis advocated is that the point of using formalisms in linguistics is more methodological than substantial and that it can be well accommodated within the conceptual framework posited by de Saussure. 1 De Saussure vs. Structuralism Before beginning to discuss structuralism, let us stress the distinction between the genuine views of Ferdinand de Saussure and the teachings of his various avowed followers, be they linguists or philosophers. In fact, de Saussure's theory, as presented in his Course, is an austere and utterly rational scientific theory articulated with a rigour commonly associated with linguistic theories of the 'post-Chomskian' period, though differing from them by the absence of -$526/$9 3(5(*5,1 formalisms. -
Reflection in the Chomsky Hierarchy
Journal of Automata, Languages and Combinatorics 18 (2013) 1, 53–60 c Otto-von-Guericke-Universit¨at Magdeburg REFLECTION IN THE CHOMSKY HIERARCHY Henk Barendregt Institute of Computing and Information Science, Radboud University, Nijmegen, The Netherlands e-mail: [email protected] Venanzio Capretta School of Computer Science, University of Nottingham, United Kingdom e-mail: [email protected] and Dexter Kozen Computer Science Department, Cornell University, Ithaca, New York 14853, USA e-mail: [email protected] ABSTRACT The class of regular languages can be generated from the regular expressions. These regular expressions, however, do not themselves form a regular language, as can be seen using the pumping lemma. On the other hand, the class of enumerable languages can be enumerated by a universal language that is one of its elements. We say that the enumerable languages are reflexive. In this paper we investigate what other classes of the Chomsky Hierarchy are re- flexive in this sense. To make this precise we require that the decoding function is itself specified by a member of the same class. Could it be that the regular languages are reflexive, by using a different collection of codes? It turns out that this is impossi- ble: the collection of regular languages is not reflexive. Similarly the collections of the context-free, context-sensitive, and computable languages are not reflexive. Therefore the class of enumerable languages is the only reflexive one in the Chomsky Hierarchy. In honor of Roel de Vrijer on the occasion of his sixtieth birthday 1. Introduction Let (Σ∗) be a class of languages over an alphabet Σ. -
Appendix A: Hierarchy of Grammar Formalisms
Appendix A: Hierarchy of Grammar Formalisms The following figure recalls the language hierarchy that we have developed in the course of the book. '' $$ '' $$ '' $$ ' ' $ $ CFG, 1-MCFG, PDA n n L1 = {a b | n ≥ 0} & % TAG, LIG, CCG, tree-local MCTAG, EPDA n n n ∗ L2 = {a b c | n ≥ 0}, L3 = {ww | w ∈{a, b} } & % 2-LCFRS, 2-MCFG, simple 2-RCG & % 3-LCFRS, 3-MCFG, simple 3-RCG n n n n n ∗ L4 = {a1 a2 a3 a4 a5 | n ≥ 0}, L5 = {www | w ∈{a, b} } & % ... LCFRS, MCFG, simple RCG, MG, set-local MCTAG, finite-copying LFG & % Thread Automata (TA) & % PMCFG 2n L6 = {a | n ≥ 0} & % RCG, simple LMG (= PTIME) & % mildly context-sensitive L. Kallmeyer, Parsing Beyond Context-Free Grammars, Cognitive Technologies, DOI 10.1007/978-3-642-14846-0, c Springer-Verlag Berlin Heidelberg 2010 216 Appendix A For each class the different formalisms and automata that generate/accept exactly the string languages contained in this class are listed. Furthermore, examples of typical languages for this class are added, i.e., of languages that belong to this class while not belonging to the next smaller class in our hier- archy. The inclusions are all proper inclusions, except for the relation between LCFRS and Thread Automata (TA). Here, we do not know whether the in- clusion is a proper one. It is possible that both devices yield the same class of languages. Appendix B: List of Acronyms The following table lists all acronyms that occur in this book. (2,2)-BRCG Binary bottom-up non-erasing RCG with at most two vari- ables per left-hand side argument 2-SA Two-Stack Automaton