DOCSLIB.ORG

Generalised LR Parsing Algorithms

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Generalised LR parsing algorithms Giorgios Robert Economopoulos Department of Computer Science Royal Holloway, University of London August, 2006 Abstract This thesis concerns the parsing of context-free grammars. A parser is a tool, de- ¯ned for a speci¯c grammar, that constructs a syntactic representation of an input string and determines if the string is grammatically correct or not. An algorithm that is capable of parsing any context-free grammar is called a generalised (context- free) parser. This thesis is devoted to the theoretical analysis of generalised parsing algorithms. We describe, analyse and compare several algorithms that are based on Knuth's LR parser. This work underpins the design and implementation of the Parser Animation Tool (PAT). We use PAT to evaluate the asymptotic complexity of generalised parsing algorithms and to develop the Binary Right Nulled Generalised LR algorithm { a new cubic worst case parser. We also compare the Right Nullable Generalised LR, Reduction Incorporated Generalised LR, Farshi, Tomita and Ear- ley algorithms using the statistical data collected by PAT. Our study indicates that the overheads associated with some of the parsing algorithms may have signi¯cant consequences on their behaviour. 2 Acknowledgements My sincere gratitude goes to Elizabeth Scott and Adrian Johnstone. Their support and guidance throughout the course of my PhD has been invaluable. I would also like to thank my examiners, Peter Rounce and Nigel Horspool, for their constructive comments and suggestions that really improved the thesis. Many friends and colleagues have helped me over the last four years, but special thanks goes to Martin Green, Rob Delicata, Jurgen Vinju, Tijs van der Storm, Magiel Bruntink, Hartwig Bosse and Urtzi Ayesta. Of course none of this would have been possible without the continued support of my parents, Maggie and Costas, and my brother, Chris. Finally, I would like to thank Valerie for being there for me when I needed her most. 3 Contents 1 Introduction 15 1.1 Motivation . 15 1.2 Contributions . 16 1.3 Outline of thesis . 17 I Background material 18 2 Recognition and parsing 19 2.1 Languages and grammars . 19 2.2 Context-free grammars . 21 2.2.1 Generating sentences . 21 2.2.2 Backus Naur form . 22 2.2.3 Recognition and parsing . 24 2.2.4 Parse trees . 24 2.3 Parsing context-free grammars . 24 2.3.1 Top-down parsing . 25 2.3.2 Bottom-up parsing . 27 2.3.3 Parsing with searching . 29 2.3.4 Ambiguous grammars . 29 2.4 Parsing deterministic context-free grammars . 30 2.4.1 Constructing a handle ¯nding automaton . 30 2.4.2 Parsing with a DFA . 35 2.4.3 LR parsing . 36 2.4.4 Parsing with lookahead . 38 2.5 Summary . 41 3 The development of generalised parsing 42 3.1 Overview . 42 3.2 Unger's method . 45 3.3 The CYK algorithm . 47 3.4 Summary . 49 4 4 Generalised LR parsing 51 4.1 Using the LR algorithm to parse all context-free grammars . 51 4.2 Tomita's Generalised LR parsing algorithm . 55 4.2.1 Graph structured stacks . 56 4.2.2 Example { constructing a GSS . 56 4.2.3 Tomita's Algorithm 1 . 58 4.2.4 Parsing with ²-rules . 61 4.2.5 Tomita's Algorithm 2 . 63 4.2.6 The non-termination of Algorithm 2 . 67 4.3 Farshi's extension of Algorithm 1 . 68 4.3.1 Example { a hidden-left recursive grammar . 70 4.3.2 Example { a hidden-right recursive grammar . 72 4.4 Constructing derivations . 75 4.4.1 Shared packed parse forests . 76 4.4.2 Tomita's Algorithm 4 . 78 4.4.3 Rekers' SPPF construction . 81 4.5 Summary . 88 II Improving Generalised LR parsers 89 5 Right Nulled Generalised LR parsing 90 5.1 Tomita's Algorithm 1e . 91 5.1.1 Example { a hidden-left recursive grammar . 93 5.1.2 Paths in the GSS . 95 5.2 Right Nulled parse tables . 98 5.3 Reducing non-determinism . 99 5.4 The RNGLR recognition algorithm . 101 5.5 The RNGLR parsing algorithm . 105 5.6 Resolvability . 112 5.7 Summary . 115 6 Binary Right Nulled Generalised LR parsing 116 6.1 The worst case complexity of GLR recognisers . 117 6.2 Achieving cubic time complexity by factoring the grammar . 121 6.3 Achieving cubic time complexity by modifying the parse table . 124 6.4 The BRNGLR recognition algorithm . 125 6.5 Performing ‘on-the-fly’ reduction path factorisation . 127 6.6 GLR parsing in at most cubic time . 131 6.7 Packing of bookkeeping SPPF nodes . 136 6.8 Summary . 141 5 7 Reduction Incorporated Generalised LR parsing 143 7.1 The role of the stack in bottom-up parsers . 144 7.2 Constructing the Intermediate Reduction Incorporated Automata . 148 7.3 Reducing non-determinism in the IRIA . 151 7.4 Regular recognition . 152 7.4.1 Recursion call automata . 152 7.4.2 Parse table representation of RCA . 154 7.5 Generalised regular recognition . 155 7.5.1 Example { ensuring all pop actions are done . 161 7.6 Reducing the non-determinism in the RCA . 166 7.7 Reducing the number of processes in each Ui .............. 166 7.8 Generalised regular parsing . 167 7.8.1 Constructing derivation trees . 167 7.8.2 Constructing an SPPF . 169 7.9 Summary . 178 8 Other approaches to generalised parsing 180 8.1 Mark-Jan Nederhof and Janos J. Sarbo . 180 8.2 James R. Kipps . 183 8.3 Jay Earley . 188 8.4 Bernard Lang . 192 8.5 Klaas Sikkel . 192 III Implementing Generalised LR parsing 194 9 Some generalised parser generators 195 9.1 ANTLR ................................... 196 9.2 PRECCx .................................. 197 9.3 JavaCC . 197 9.4 BtYacc . 197 9.5 Bison . 198 9.6 Elkhound: a GLR parser generator . 198 9.7 Asf+Sdf Meta-Environment . 199 10 Experimental investigation 201 10.1 Overview of chapter . 201 10.2 The Grammar Tool Box and Parser Animation Tool . 202 10.3 The grammars . 202 10.4 The input strings . 203 10.5 Parse table sizes . 204 6 10.6 The performance of the RNGLR algorithm compared to the Tomita and Farshi algorithms . 207 10.6.1 A non-cubic example . 207 10.6.2 Using di®erent LR tables . 209 10.6.3 The performance of Farshi's algorithms . 211 10.7 The performance of the BRNGLR algorithm . 214 10.7.1 GSS construction . 214 10.7.2 SPPF construction . 218 10.8 The performance of Earley algorithm . 222 10.9 The performance of the RIGLR algorithm . 224 10.9.1 The size of RIA and RCA . 224 10.9.2 Asymptotic behaviour . 225 IV 229 11 Concluding remarks 230 11.1 Conclusions . 230 11.2 Future work . 231 Bibliography 233 A Parser Animation Tool user manual 241 A.1 Acquisition and deployment of PAT and its associated tools . 241 A.1.1 Java Runtime Environment (JRE) . 241 A.1.2 Downloading PAT . 241 A.1.3 The Grammar Tool Box . 242 A.1.4 aiSee . 242 A.1.5 Input library . 242 A.2 User interface . 243 A.2.1 Specifying the grammar, parse table and string to parse . 244 A.2.2 Selecting a parsing algorithm . 245 A.2.3 Output options . 247 A.3 Animating parses . 248 A.3.1 GDL output of GSS and SPPF . 249 A.3.2 Histograms . 250 A.3.3.
Recommended publications
  • The Earley Algorithm Is to Avoid This, by Only Building Constituents That Are Compatible with the Input Read So Far

    The Earley Algorithm Is to Avoid This, by Only Building Constituents That Are Compatible with the Input Read So Far

    Earley Parsing Informatics 2A: Lecture 21 Shay Cohen 3 November 2017 1 / 31 1 The CYK chart as a graph What's wrong with CYK Adding Prediction to the Chart 2 The Earley Parsing Algorithm The Predictor Operator The Scanner Operator The Completer Operator Earley parsing: example Comparing Earley and CYK 2 / 31 We would have to split a given span into all possible subspans according to the length of the RHS. What is the complexity of such algorithm? Still O(n2) charts, but now it takes O(nk−1) time to process each cell, where k is the maximal length of an RHS. Therefore: O(nk+1). For CYK, k = 2. Can we do better than that? Note about CYK The CYK algorithm parses input strings in Chomsky normal form. Can you see how to change it to an algorithm with an arbitrary RHS length (of only nonterminals)? 3 / 31 Still O(n2) charts, but now it takes O(nk−1) time to process each cell, where k is the maximal length of an RHS. Therefore: O(nk+1). For CYK, k = 2. Can we do better than that? Note about CYK The CYK algorithm parses input strings in Chomsky normal form. Can you see how to change it to an algorithm with an arbitrary RHS length (of only nonterminals)? We would have to split a given span into all possible subspans according to the length of the RHS. What is the complexity of such algorithm? 3 / 31 Note about CYK The CYK algorithm parses input strings in Chomsky normal form.
  • LATE Ain't Earley: a Faster Parallel Earley Parser

    LATE Ain't Earley: a Faster Parallel Earley Parser

    LATE Ain’T Earley: A Faster Parallel Earley Parser Peter Ahrens John Feser Joseph Hui [email protected] [email protected] [email protected] July 18, 2018 Abstract We present the LATE algorithm, an asynchronous variant of the Earley algorithm for pars- ing context-free grammars. The Earley algorithm is naturally task-based, but is difficult to parallelize because of dependencies between the tasks. We present the LATE algorithm, which uses additional data structures to maintain information about the state of the parse so that work items may be processed in any order. This property allows the LATE algorithm to be sped up using task parallelism. We show that the LATE algorithm can achieve a 120x speedup over the Earley algorithm on a natural language task. 1 Introduction Improvements in the efficiency of parsers for context-free grammars (CFGs) have the potential to speed up applications in software development, computational linguistics, and human-computer interaction. The Earley parser has an asymptotic complexity that scales with the complexity of the CFG, a unique, desirable trait among parsers for arbitrary CFGs. However, while the more commonly used Cocke-Younger-Kasami (CYK) [2, 5, 12] parser has been successfully parallelized [1, 7], the Earley algorithm has seen relatively few attempts at parallelization. Our research objectives were to understand when there exists parallelism in the Earley algorithm, and to explore methods for exploiting this parallelism. We first tried to naively parallelize the Earley algorithm by processing the Earley items in each Earley set in parallel. We found that this approach does not produce any speedup, because the dependencies between Earley items force much of the work to be performed sequentially.
  • A Declarative Extension of Parsing Expression Grammars for Recognizing Most Programming Languages

    A Declarative Extension of Parsing Expression Grammars for Recognizing Most Programming Languages

    Journal of Information Processing Vol.24 No.2 256–264 (Mar. 2016) [DOI: 10.2197/ipsjjip.24.256] Regular Paper A Declarative Extension of Parsing Expression Grammars for Recognizing Most Programming Languages Tetsuro Matsumura1,†1 Kimio Kuramitsu1,a) Received: May 18, 2015, Accepted: November 5, 2015 Abstract: Parsing Expression Grammars are a popular foundation for describing syntax. Unfortunately, several syn- tax of programming languages are still hard to recognize with pure PEGs. Notorious cases appears: typedef-defined names in C/C++, indentation-based code layout in Python, and HERE document in many scripting languages. To recognize such PEG-hard syntax, we have addressed a declarative extension to PEGs. The “declarative” extension means no programmed semantic actions, which are traditionally used to realize the extended parsing behavior. Nez is our extended PEG language, including symbol tables and conditional parsing. This paper demonstrates that the use of Nez Extensions can realize many practical programming languages, such as C, C#, Ruby, and Python, which involve PEG-hard syntax. Keywords: parsing expression grammars, semantic actions, context-sensitive syntax, and case studies on program- ming languages invalidate the declarative property of PEGs, thus resulting in re- 1. Introduction duced reusability of grammars. As a result, many developers need Parsing Expression Grammars [5], or PEGs, are a popu- to redevelop grammars for their software engineering tools. lar foundation for describing programming language syntax [6], In this paper, we propose a declarative extension of PEGs for [15]. Indeed, the formalism of PEGs has many desirable prop- recognizing context-sensitive syntax. The “declarative” exten- erties, including deterministic behaviors, unlimited look-aheads, sion means no arbitrary semantic actions that are written in a gen- and integrated lexical analysis known as scanner-less parsing.
  • Lecture 10: CYK and Earley Parsers Alvin Cheung Building Parse Trees Maaz Ahmad CYK and Earley Algorithms Talia Ringer More Disambiguation

    Lecture 10: CYK and Earley Parsers Alvin Cheung Building Parse Trees Maaz Ahmad CYK and Earley Algorithms Talia Ringer More Disambiguation

    Hack Your Language! CSE401 Winter 2016 Introduction to Compiler Construction Ras Bodik Lecture 10: CYK and Earley Parsers Alvin Cheung Building Parse Trees Maaz Ahmad CYK and Earley algorithms Talia Ringer More Disambiguation Ben Tebbs 1 Announcements • HW3 due Sunday • Project proposals due tonight – No late days • Review session this Sunday 6-7pm EEB 115 2 Outline • Last time we saw how to construct AST from parse tree • We will now discuss algorithms for generating parse trees from input strings 3 Today CYK parser builds the parse tree bottom up More Disambiguation Forcing the parser to select the desired parse tree Earley parser solves CYK’s inefficiency 4 CYK parser Parser Motivation • Given a grammar G and an input string s, we need an algorithm to: – Decide whether s is in L(G) – If so, generate a parse tree for s • We will see two algorithms for doing this today – Many others are available – Each with different tradeoffs in time and space 6 CYK Algorithm • Parsing algorithm for context-free grammars • Invented by John Cocke, Daniel Younger, and Tadao Kasami • Basic idea given string s with n tokens: 1. Find production rules that cover 1 token in s 2. Use 1. to find rules that cover 2 tokens in s 3. Use 2. to find rules that cover 3 tokens in s 4. … N. Use N-1. to find rules that cover n tokens in s. If succeeds then s is in L(G), else it is not 7 A graphical way to visualize CYK Initial graph: the input (terminals) Repeat: add non-terminal edges until no more can be added.
  • Efficient Recursive Parsing

    Efficient Recursive Parsing

    Purdue University Purdue e-Pubs Department of Computer Science Technical Reports Department of Computer Science 1976 Efficient Recursive Parsing Christoph M. Hoffmann Purdue University, [email protected] Report Number: 77-215 Hoffmann, Christoph M., "Efficient Recursive Parsing" (1976). Department of Computer Science Technical Reports. Paper 155. https://docs.lib.purdue.edu/cstech/155 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact [email protected] for additional information. EFFICIENT RECURSIVE PARSING Christoph M. Hoffmann Computer Science Department Purdue University West Lafayette, Indiana 47907 CSD-TR 215 December 1976 Efficient Recursive Parsing Christoph M. Hoffmann Computer Science Department Purdue University- Abstract Algorithms are developed which construct from a given LL(l) grammar a recursive descent parser with as much, recursion resolved by iteration as is possible without introducing auxiliary memory. Unlike other proposed methods in the literature designed to arrive at parsers of this kind, the algorithms do not require extensions of the notational formalism nor alter the grammar in any way. The algorithms constructing the parsers operate in 0(k«s) steps, where s is the size of the grammar, i.e. the sum of the lengths of all productions, and k is a grammar - dependent constant. A speedup of the algorithm is possible which improves the bound to 0(s) for all LL(l) grammars, and constructs smaller parsers with some auxiliary memory in form of parameters
  • A Mobile App for Teaching Formal Languages and Automata

    A Mobile App for Teaching Formal Languages and Automata

    Received: 21 December 2017 | Accepted: 19 March 2018 DOI: 10.1002/cae.21944 SPECIAL ISSUE ARTICLE A mobile app for teaching formal languages and automata Carlos H. Pereira | Ricardo Terra Department of Computer Science, Federal University of Lavras, Lavras, Brazil Abstract Formal Languages and Automata (FLA) address mathematical models able to Correspondence Ricardo Terra, Department of Computer specify and recognize languages, their properties and characteristics. Although Science, Federal University of Lavras, solid knowledge of FLA is extremely important for a B.Sc. degree in Computer Postal Code 3037, Lavras, Brazil. Science and similar fields, the algorithms and techniques covered in the course Email: [email protected] are complex and difficult to assimilate. Therefore, this article presents FLApp, Funding information a mobile application—which we consider the new way to reach students—for FAPEMIG (Fundação de Amparo à teaching FLA. The application—developed for mobile phones and tablets Pesquisa do Estado de Minas Gerais) running Android—provides students not only with answers to problems involving Regular, Context-free, Context-Sensitive, and Recursively Enumer- able Languages, but also an Educational environment that describes and illustrates each step of the algorithms to support students in the learning process. KEYWORDS automata, education, formal languages, mobile application 1 | INTRODUCTION In this article, we present FLApp (Formal Languages and Automata Application), a mobile application for teaching FLA Formal Languages and Automata (FLA) is an important area that helps students by solving problems involving Regular, of Computer Science that approaches mathematical models Context-free, Context-Sensitive, and Recursively Enumerable able to specify and recognize languages, their properties and Languages (levels 3 to 0, respectively), in addition to create an characteristics [14].
  • An Earley Parsing Algorithm for Range Concatenation Grammars

    An Earley Parsing Algorithm for Range Concatenation Grammars

    An Earley Parsing Algorithm for Range Concatenation Grammars Laura Kallmeyer Wolfgang Maier Yannick Parmentier SFB 441 SFB 441 CNRS - LORIA Universitat¨ Tubingen¨ Universitat¨ Tubingen¨ Nancy Universite´ 72074 Tubingen,¨ Germany 72074 Tubingen,¨ Germany 54506 Vandœuvre, France [email protected] [email protected] [email protected] Abstract class of LCFRS has received more attention con- cerning parsing (Villemonte de la Clergerie, 2002; We present a CYK and an Earley-style Burden and Ljunglof,¨ 2005). This article proposes algorithm for parsing Range Concatena- new CYK and Earley parsers for RCG, formulat- tion Grammar (RCG), using the deduc- ing them in the framework of parsing as deduction tive parsing framework. The characteris- (Shieber et al., 1995). The second section intro- tic property of the Earley parser is that we duces necessary definitions. Section 3 presents a use a technique of range boundary con- CYK-style algorithm and Section 4 extends this straint propagation to compute the yields with an Earley-style prediction. of non-terminals as late as possible. Ex- periments show that, compared to previ- 2 Preliminaries ous approaches, the constraint propagation The rules (clauses) of RCGs1 rewrite predicates helps to considerably decrease the number ranging over parts of the input by other predicates. of items in the chart. E.g., a clause S(aXb) S(X) signifies that S is → 1 Introduction true for a part of the input if this part starts with an a, ends with a b, and if, furthermore, S is also true RCGs (Boullier, 2000) have recently received a for the part between a and b.
  • Advanced Parsing Techniques

    Advanced Parsing Techniques

    Advanced Parsing Techniques Announcements ● Written Set 1 graded. ● Hard copies available for pickup right now. ● Electronic submissions: feedback returned later today. Where We Are Where We Are Parsing so Far ● We've explored five deterministic parsing algorithms: ● LL(1) ● LR(0) ● SLR(1) ● LALR(1) ● LR(1) ● These algorithms all have their limitations. ● Can we parse arbitrary context-free grammars? Why Parse Arbitrary Grammars? ● They're easier to write. ● Can leave operator precedence and associativity out of the grammar. ● No worries about shift/reduce or FIRST/FOLLOW conflicts. ● If ambiguous, can filter out invalid trees at the end. ● Generate candidate parse trees, then eliminate them when not needed. ● Practical concern for some languages. ● We need to have C and C++ compilers! Questions for Today ● How do you go about parsing ambiguous grammars efficiently? ● How do you produce all possible parse trees? ● What else can we do with a general parser? The Earley Parser Motivation: The Limits of LR ● LR parsers use shift and reduce actions to reduce the input to the start symbol. ● LR parsers cannot deterministically handle shift/reduce or reduce/reduce conflicts. ● However, they can nondeterministically handle these conflicts by guessing which option to choose. ● What if we try all options and see if any of them work? The Earley Parser ● Maintain a collection of Earley items, which are LR(0) items annotated with a start position. ● The item A → α·ω @n means we are working on recognizing A → αω, have seen α, and the start position of the item was the nth token. ● Using techniques similar to LR parsing, try to scan across the input creating these items.
  • Verifiable Parse Table Composition for Deterministic Parsing*

    Verifiable Parse Table Composition for Deterministic Parsing*

    Verifiable Parse Table Composition for Deterministic Parsing? August Schwerdfeger and Eric Van Wyk Department of Computer Science and Engineering University of Minnesota, Minneapolis, MN {schwerdf,evw}@cs.umn.edu Abstract. One obstacle to the implementation of modular extensions to programming languages lies in the problem of parsing extended lan- guages. Specifically, the parse tables at the heart of traditional LALR(1) parsers are so monolithic and tightly constructed that, in the general case, it is impossible to extend them without regenerating them from the source grammar. Current extensible frameworks employ a variety of solutions, ranging from a full regeneration to using pluggable binary mod- ules for each different extension. But recompilation is time-consuming, while the pluggable modules in many cases cannot support the addition of more than one extension, or use backtracking or non-deterministic parsing techniques. We present here a middle-ground approach that allows an extension, if it meets certain restrictions, to be compiled into a parse table fragment. The host language parse table and fragments from multiple extensions can then always be efficiently composed to produce a conflict-free parse table for the extended language. This allows for the distribution of de- terministic parsers for extensible languages in a pre-compiled format, eliminating the need for the “source code” grammar to be distributed. In practice, we have found these restrictions to be reasonable and admit many useful language extensions. 1 Introduction In parsing programming languages, the usual practice is to generate a single parser for the language to be parsed. A well known and often-used approach is LR parsing [1] which relies on a process, sometimes referred to as grammar compilation, to generate a monolithic parse table representing the grammar being parsed.
  • Basics of Compiler Design

    Basics of Compiler Design

    Basics of Compiler Design Anniversary edition Torben Ægidius Mogensen DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF COPENHAGEN Published through lulu.com. c Torben Ægidius Mogensen 2000 – 2010 [email protected] Department of Computer Science University of Copenhagen Universitetsparken 1 DK-2100 Copenhagen DENMARK Book homepage: http://www.diku.dk/∼torbenm/Basics First published 2000 This edition: August 20, 2010 ISBN 978-87-993154-0-6 Contents 1 Introduction 1 1.1 What is a compiler? . 1 1.2 The phases of a compiler . 2 1.3 Interpreters . 3 1.4 Why learn about compilers? . 4 1.5 The structure of this book . 5 1.6 To the lecturer . 6 1.7 Acknowledgements . 7 1.8 Permission to use . 7 2 Lexical Analysis 9 2.1 Introduction . 9 2.2 Regular expressions . 10 2.2.1 Shorthands . 13 2.2.2 Examples . 14 2.3 Nondeterministic finite automata . 15 2.4 Converting a regular expression to an NFA . 18 2.4.1 Optimisations . 20 2.5 Deterministic finite automata . 22 2.6 Converting an NFA to a DFA . 23 2.6.1 Solving set equations . 23 2.6.2 The subset construction . 26 2.7 Size versus speed . 29 2.8 Minimisation of DFAs . 30 2.8.1 Example . 32 2.8.2 Dead states . 34 2.9 Lexers and lexer generators . 35 2.9.1 Lexer generators . 41 2.10 Properties of regular languages . 42 2.10.1 Relative expressive power . 42 2.10.2 Limits to expressive power . 44 i ii CONTENTS 2.10.3 Closure properties . 45 2.11 Further reading .
  • Deterministic Parsing of Languages with Dynamic Operators

    Deterministic Parsing of Languages with Dynamic Operators

    Deterministic Parsing of Languages with Dynamic Op erators Kjell Post Allen Van Gelder James Kerr Baskin Computer Science Center University of California Santa Cruz, CA 95064 email: [email protected] UCSC-CRL-93-15 July 27, 1993 Abstract Allowing the programmer to de ne op erators in a language makes for more readable co de but also complicates the job of parsing; standard parsing techniques cannot accommo date dynamic grammars. We presentan LR parsing metho dology, called deferreddecision parsing , that handles dynamic op erator declarations, that is, op erators that are declared at run time, are applicable only within a program or context, and are not in the underlying language or grammar. It uses a parser generator that takes pro duction rules as input, and generates a table-driven LR parser, much like yacc. Shift/reduce con icts that involve dynamic op erators are resolved at parse time rather than at table construction time. For an op erator-rich language, this technique reduces the size of the grammar needed and parse table pro duced. The added cost to the parser is minimal. Ambiguous op erator constructs can either b e detected by the parser as input is b eing read or avoided altogether by enforcing reasonable restrictions on op erator declarations. Wehave b een able to describ e the syntax of Prolog, a language known for its lib eral use of op erators, and Standard ML, which supp orts lo cal declarations of op erators. De nite clause grammars DCGs, a novel parsing feature of Prolog, can b e translated into ecient co de by our parser generator.
  • Principles and Implementation of Deductive Parsing

    Principles and Implementation of Deductive Parsing

    Principles and implementation of deductive parsing The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Stuart M. Shieber, Yves Schabes, and Fernando C. N. Pereira. Principles and implementation of deductive parsing. Journal of Logic Programming, 24(1-2):3-36, July-August 1995. Also available as cmp-lg/9404008. Published Version http://dx.doi.org/10.1016/0743-1066(95)00035-I Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:2031716 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA J. LOGIC PROGRAMMING 1995:24(1{2):3{36 3 PRINCIPLES AND IMPLEMENTATION OF DEDUCTIVE PARSING STUART M. SHIEBER, YVES SCHABES, ∗ AND FERNANDO C. N. PEREIRAy £ We present a system for generating parsers based directly on the metaphor of parsing as deduction. Parsing algorithms can be represented directly as deduction systems, and a single deduction engine can interpret such de- duction systems so as to implement the corresponding parser. The method generalizes easily to parsers for augmented phrase structure formalisms, such as definite-clause grammars and other logic grammar formalisms, and has been used for rapid prototyping of parsing algorithms for a variety of formalisms including variants of tree-adjoining grammars, categorial gram- mars, and lexicalized context-free grammars. ¡ 1. INTRODUCTION Parsing can be viewed as a deductive process that seeks to prove claims about the grammatical status of a string from assumptions describing the grammatical properties of the string's elements and the linear order between them.