DOCSLIB.ORG

Top Down Parsing Example Example #2 Solutions?

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Top Down Parsing Example Example #2 Solutions? Top Down Parsing Example Suppose we have a grammar: A legal parse would be: . When we are parsing, we produce a unique syntax tree from a legal 1. S 1 + 2 * 3 sentence. S → E only one rule: S → E —An unambiguous grammar gives rise to a single leftmost E → T | E+T 2. E 1 + 2 * 3 derivation for any sentence in the language. T → F | T*F choose E → E+T . So, if we are trying to recognise a sentence, what we are trying to do F → unit | (E) 3. E+T 1 + 2 * 3 is grow a parse tree corresponding to that sentence choose E → T → F→ unit —We are trying to find the leftmost derivation. and the expression: 4. unit+T 1 + 2 * 3 . A top-down parser constructs a leftmost parse 1 + 2 * 3 match 1 and + —We will always be looking at the leftmost nonterminal. 5. 1+T 2 * 3 This means that to choose T → F→ unit . This follows the push down automaton model of the previous lecture parse this sentence 6. 1+unit 2 * 3 . The parser must choose the correct production from the set of some backtracking is match 2 productions that correspond to the current state of the parse. required, i.e., put input 7. 1+2 *3 . If at any time there is no candidate production corresponding to the symbols back! ✸WRONG state of the parse, we must have made a wrong turn at some earlier Backtracking in stage and we will need to backtrack. compilers is nontrivial and to be avoided!! CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 1 CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 2 Example #2 Solutions? Suppose we have a grammar: A legal parse would be: . We could rearrange the productions so that the left recursive ones 1. S 1 + 2 * 3 come at the end, and always choose the first matching production. S → E only one rule: S → E . For the previous examples, this has already been done. The left E → T | E+T 2. E 1 + 2 * 3 recursive ones are at the end of the list! T → F | T*F choose E → E+T . Note that this is not an easy task in general since mutually F → unit | (E) 3. E+T 1 + 2 * 3 recursive grammars have the same problems: choose E → E+T and the expression: 4. E+T+T 1 + 2 * 3 A → B | C D 1 + 2 * 3 choose E → E+T C → E | A F 5. E+T+T+T 1 + 2 * 3 . In general, rearranging productions will not help – the parser will choose E → E+T still have problems. ...you can see what happens! —Even if it does help, a parser which needs to backtrack an arbitrary distance is inefficient. The problem is simple: Left Recursion! . What we need is a way to deterministically parse a grammar in a top down fashion without backtracking. CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 3 CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 4 1 Eliminating left recursion Example of eliminating left recursion . An algorithm to eliminate arbitrary left recursion (by replacing it . Consider the productions: with right recursion) is as follows: A → a | Ba B → b | Cb C → c | Ac 1. Arbitrarily order the non-terminals: N1, N2, N3, ... 1. Arbitrarily order the non-terminals: A, B, C 2. Apply the following steps to the productions for N1, then N2, ... 2. Consider the productions for A: no change 3. For Ni: 2. Consider the productions for B: no change a) For all productions Ni → Nk α, where k < i and if the productions 3. Consider the productions for C: for Nk are Nk → β1 | β2 | β3 | ... then expand the reference to Nk, a) Replace C → Ac by C → ac | Bac i.e. replace the production N N by N ... i → k α i → β1 α | β2 α | a) Replace C → Bac by C → bac | Cbac b) If the productions for N are now i Productions for C are now: C → c | ac | bac | Cbac Ni → α1 | α2 | ... | Ni β1 | Ni β2 | ... (where the first few are not left recursive while the latter are) b) Replace the productions for C by: then replace them with C → cC’ | acC’ | bacC’ Ni → α1 Ni’ | α2 Ni’ | ... C’ → ε | bacC’ Ni’ → ε | β1 Ni’ | β2 Ni’ | ... CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 5 CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 6 A Workable Solution Definitions Observation . A parser which can make a deterministic decision about which . The trouble which gives rise to nondeterminacy and alternative to choose when faced with one, if given a buffer of k backtracking in top down parsers shows itself in only one symbols, is called a LL(k) parser. place – that is when a parser has to choose between —Left to right scan of input several alternatives with the same left hand side. —Left most derivation . The only information which we can use to make the —k symbols of look-ahead correct decision is the input stream itself. The grammar that an LL(k) parser recognizes is an LL(k) grammar and any language that has an LL(k) grammar is an LL(k) language. —In the example, we (humans) could see which alternative to choose by looking at the input yet-to-be-read. —We are constructing an LL(1) compiler that recognises LL(1) grammars. If we are going to look ahead in order to make the correct —So the question is How do we know when we have an LL(1) decision, we need a buffer in which to store the next few grammar? symbols. We also have LR(k) grammars and other variations, but our focus is . In practice, this buffer is of a fixed length. currently on LL(1) grammars. CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 7 CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 8 2 Definition of LL(1) Definition of First . When faced with a production such as: . To compute FIRST(X) for all grammar symbols X, apply the following algorithm until no more terminals or can be A → α | α | α ε 1 2 3 added to any FIRST set. We chose one of the α uniquely by looking at the i 1. If X is a terminal, then FIRST(X) is {X} next input symbol. 2. If X → ε is a production, then add ε to FIRST(X) . We employ two sets: first and follow, to help us. 3. If X is a nonterminal and X → Y1 Y2 ...Yn is a Recall: production, then place a in FIRST(X) if for some i, a is in . First(X) is the set of all terminal symbols that can FIRST(Yi), and ε is in all of FIRST(Y1 ), ..., FIRST (Yi-1 ); “start” the production X that is Y1 Y2 ...Yi-1 ⇒ * ε . If ε is in FIRST(Yj ) for all j = 1, 2, ... n, then add ε to . Follow(X) is the set of terminal symbols that can FIRST(X). For example, everything in FIRST(Y ) is surely follow an “X” 1 in FIRST(X). If Y1 does not derive ε, then we add nothing more to FIRST(X), but if Y1 ⇒ ε then we add FIRST(Y2) and so on. CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 9 CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 10 Definition of Follow Definition of LL(1) property . To compute FOLLOW(A) for all nonterminals A, apply the following algorithm until nothing can be added to any FOLLOW set. 1. Place $ in FOLLOW(S), where S is the start symbol and Definition: A grammar G is LL(1) if and only if for all rules $ is the input right endmarker. A → α1 | α2 | ... | αn 2. If there is a production A → α B β then everything in . director(αi) ∩ director(αk) = ∅ ∀ i ≠ k FIRST(β) except for ε is placed in FOLLOW(B). where: ∗ director( ) = first( ) follow(A) if 3. If there is a production A → αB or a production A → α Bβ αi αi ∪ αj ⇒ ε = first(α ) otherwise where FIRST(β) contains ε (i.e., β ⇒ * ε), then i everything in FOLLOW(A) is in FOLLOW(B). CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 11 CC&P 2004 © 2003, 2004 Kevin J. Maciunas, Charles Lakos Slide 12 3 Making Grammars LL(1) Fixing the problem S → T . We can't always make a grammar which is not LL(1) into T → L B | L C array 1 an equivalent LL(1) grammar. L → long | ε . Some tricks to help are factorisation and substitution. C → B | ε Transform the grammar by factorisation: B → real | integer S → T Consider the grammar T → L X S → T X → B | C array T → L B | L C array 2 L → long | ε L → long | ε C → B | ε C → B | ε Transform the grammar by factorisation: B → real | integer B → real | integer S → T Substitute for C wherever it occurs: T → L X S → T Factorisation of B in X gives: X → B | C array T → L X 3 S → T L → long | ε X → B | B array | array T → L X C → B | ε L → long | ε X → B Y | array B → real | integer B → real | integer Y → array | ε L → long | ε ✸ Note that it still is not LL(1)! B → real | integer CC&P 2004 © 2003, 2004 Kevin J.
Load more

Recommended publications
  • Syntax-Directed Translation, Parse Trees, Abstract Syntax Trees
    Syntax-Directed Translation Extending CFGs Grammar Annotation Parse Trees Abstract Syntax Trees (ASTs) Readings: Section 5.1, 5.2, 5.5, 5.6 Motivation: parser as a translator syntax-directed translation stream of ASTs, or tokens parser assembly code syntax + translation rules (typically hardcoded in the parser) 1 Mechanism of syntax-directed translation syntax-directed translation is done by extending the CFG a translation rule is defined for each production given X Æ d A B c the translation of X is defined in terms of translation of nonterminals A, B values of attributes of terminals d, c constants To translate an input string: 1. Build the parse tree. 2. Working bottom-up • Use the translation rules to compute the translation of each nonterminal in the tree Result: the translation of the string is the translation of the parse tree's root nonterminal Why bottom up? a nonterminal's value may depend on the value of the symbols on the right-hand side, so translate a non-terminal node only after children translations are available 2 Example 1: arith expr to its value Syntax-directed translation: the CFG translation rules E Æ E + T E1.trans = E2.trans + T.trans E Æ T E.trans = T.trans T Æ T * F T1.trans = T2.trans * F.trans T Æ F T.trans = F.trans F Æ int F.trans = int.value F Æ ( E ) F.trans = E.trans Example 1 (cont) E (18) Input: 2 * (4 + 5) T (18) T (2) * F (9) F (2) ( E (9) ) int (2) E (4) * T (5) Annotated Parse Tree T (4) F (5) F (4) int (5) int (4) 3 Example 2: Compute type of expr E -> E + E if ((E2.trans == INT) and (E3.trans == INT) then E1.trans = INT else E1.trans = ERROR E -> E and E if ((E2.trans == BOOL) and (E3.trans == BOOL) then E1.trans = BOOL else E1.trans = ERROR E -> E == E if ((E2.trans == E3.trans) and (E2.trans != ERROR)) then E1.trans = BOOL else E1.trans = ERROR E -> true E.trans = BOOL E -> false E.trans = BOOL E -> int E.trans = INT E -> ( E ) E1.trans = E2.trans Example 2 (cont) Input: (2 + 2) == 4 1.
    [Show full text]
  • Derivatives of Parsing Expression Grammars
    Derivatives of Parsing Expression Grammars Aaron Moss Cheriton School of Computer Science University of Waterloo Waterloo, Ontario, Canada [email protected] This paper introduces a new derivative parsing algorithm for recognition of parsing expression gram- mars. Derivative parsing is shown to have a polynomial worst-case time bound, an improvement on the exponential bound of the recursive descent algorithm. This work also introduces asymptotic analysis based on inputs with a constant bound on both grammar nesting depth and number of back- tracking choices; derivative and recursive descent parsing are shown to run in linear time and constant space on this useful class of inputs, with both the theoretical bounds and the reasonability of the in- put class validated empirically. This common-case constant memory usage of derivative parsing is an improvement on the linear space required by the packrat algorithm. 1 Introduction Parsing expression grammars (PEGs) are a parsing formalism introduced by Ford [6]. Any LR(k) lan- guage can be represented as a PEG [7], but there are some non-context-free languages that may also be represented as PEGs (e.g. anbncn [7]). Unlike context-free grammars (CFGs), PEGs are unambiguous, admitting no more than one parse tree for any grammar and input. PEGs are a formalization of recursive descent parsers allowing limited backtracking and infinite lookahead; a string in the language of a PEG can be recognized in exponential time and linear space using a recursive descent algorithm, or linear time and space using the memoized packrat algorithm [6]. PEGs are formally defined and these algo- rithms outlined in Section 3.
    [Show full text]
  • Lecture 4 Dynamic Programming
    1/17 Lecture 4 Dynamic Programming Last update: Jan 19, 2021 References: Algorithms, Jeff Erickson, Chapter 3. Algorithms, Gopal Pandurangan, Chapter 6. Dynamic Programming 2/17 Backtracking is incredible powerful in solving all kinds of hard prob- lems, but it can often be very slow; usually exponential. Example: Fibonacci numbers is defined as recurrence: 0 if n = 0 Fn =8 1 if n = 1 > Fn 1 + Fn 2 otherwise < ¡ ¡ > A direct translation in:to recursive program to compute Fibonacci number is RecFib(n): if n=0 return 0 if n=1 return 1 return RecFib(n-1) + RecFib(n-2) Fibonacci Number 3/17 The recursive program has horrible time complexity. How bad? Let's try to compute. Denote T(n) as the time complexity of computing RecFib(n). Based on the recursion, we have the recurrence: T(n) = T(n 1) + T(n 2) + 1; T(0) = T(1) = 1 ¡ ¡ Solving this recurrence, we get p5 + 1 T(n) = O(n); = 1.618 2 So the RecFib(n) program runs at exponential time complexity. RecFib Recursion Tree 4/17 Intuitively, why RecFib() runs exponentially slow. Problem: redun- dant computation! How about memorize the intermediate computa- tion result to avoid recomputation? Fib: Memoization 5/17 To optimize the performance of RecFib, we can memorize the inter- mediate Fn into some kind of cache, and look it up when we need it again. MemFib(n): if n = 0 n = 1 retujrjn n if F[n] is undefined F[n] MemFib(n-1)+MemFib(n-2) retur n F[n] How much does it improve upon RecFib()? Assuming accessing F[n] takes constant time, then at most n additions will be performed (we never recompute).
    [Show full text]
  • Exhaustive Recursion and Backtracking
    CS106B Handout #19 J Zelenski Feb 1, 2008 Exhaustive recursion and backtracking In some recursive functions, such as binary search or reversing a file, each recursive call makes just one recursive call. The "tree" of calls forms a linear line from the initial call down to the base case. In such cases, the performance of the overall algorithm is dependent on how deep the function stack gets, which is determined by how quickly we progress to the base case. For reverse file, the stack depth is equal to the size of the input file, since we move one closer to the empty file base case at each level. For binary search, it more quickly bottoms out by dividing the remaining input in half at each level of the recursion. Both of these can be done relatively efficiently. Now consider a recursive function such as subsets or permutation that makes not just one recursive call, but several. The tree of function calls has multiple branches at each level, which in turn have further branches, and so on down to the base case. Because of the multiplicative factors being carried down the tree, the number of calls can grow dramatically as the recursion goes deeper. Thus, these exhaustive recursion algorithms have the potential to be very expensive. Often the different recursive calls made at each level represent a decision point, where we have choices such as what letter to choose next or what turn to make when reading a map. Might there be situations where we can save some time by focusing on the most promising options, without committing to exploring them all? In some contexts, we have no choice but to exhaustively examine all possibilities, such as when trying to find some globally optimal result, But what if we are interested in finding any solution, whichever one that works out first? At each decision point, we can choose one of the available options, and sally forth, hoping it works out.
    [Show full text]
  • Backtrack Parsing Context-Free Grammar Context-Free Grammar
    Context-free Grammar Problems with Regular Context-free Grammar Language and Is English a regular language? Bad question! We do not even know what English is! Two eggs and bacon make(s) a big breakfast Backtrack Parsing Can you slide me the salt? He didn't ought to do that But—No! Martin Kay I put the wine you brought in the fridge I put the wine you brought for Sandy in the fridge Should we bring the wine you put in the fridge out Stanford University now? and University of the Saarland You said you thought nobody had the right to claim that they were above the law Martin Kay Context-free Grammar 1 Martin Kay Context-free Grammar 2 Problems with Regular Problems with Regular Language Language You said you thought nobody had the right to claim [You said you thought [nobody had the right [to claim that they were above the law that [they were above the law]]]] Martin Kay Context-free Grammar 3 Martin Kay Context-free Grammar 4 Problems with Regular Context-free Grammar Language Nonterminal symbols ~ grammatical categories Is English mophology a regular language? Bad question! We do not even know what English Terminal Symbols ~ words morphology is! They sell collectables of all sorts Productions ~ (unordered) (rewriting) rules This concerns unredecontaminatability Distinguished Symbol This really is an untiable knot. But—Probably! (Not sure about Swahili, though) Not all that important • Terminals and nonterminals are disjoint • Distinguished symbol Martin Kay Context-free Grammar 5 Martin Kay Context-free Grammar 6 Context-free Grammar Context-free
    [Show full text]
  • Parsing 1. Grammars and Parsing 2. Top-Down and Bottom-Up Parsing 3
    Syntax Parsing syntax: from the Greek syntaxis, meaning “setting out together or arrangement.” 1. Grammars and parsing Refers to the way words are arranged together. 2. Top-down and bottom-up parsing Why worry about syntax? 3. Chart parsers • The boy ate the frog. 4. Bottom-up chart parsing • The frog was eaten by the boy. 5. The Earley Algorithm • The frog that the boy ate died. • The boy whom the frog was eaten by died. Slide CS474–1 Slide CS474–2 Grammars and Parsing Need a grammar: a formal specification of the structures allowable in Syntactic Analysis the language. Key ideas: Need a parser: algorithm for assigning syntactic structure to an input • constituency: groups of words may behave as a single unit or phrase sentence. • grammatical relations: refer to the subject, object, indirect Sentence Parse Tree object, etc. Beavis ate the cat. S • subcategorization and dependencies: refer to certain kinds of relations between words and phrases, e.g. want can be followed by an NP VP infinitive, but find and work cannot. NAME V NP All can be modeled by various kinds of grammars that are based on ART N context-free grammars. Beavis ate the cat Slide CS474–3 Slide CS474–4 CFG example CFG’s are also called phrase-structure grammars. CFG’s Equivalent to Backus-Naur Form (BNF). A context free grammar consists of: 1. S → NP VP 5. NAME → Beavis 1. a set of non-terminal symbols N 2. VP → V NP 6. V → ate 2. a set of terminal symbols Σ (disjoint from N) 3.
    [Show full text]
  • Abstract Syntax Trees & Top-Down Parsing
    Abstract Syntax Trees & Top-Down Parsing Review of Parsing • Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree • Issues: – How do we recognize that s ∈ L(G) ? – A parse tree of s describes how s ∈ L(G) – Ambiguity: more than one parse tree (possible interpretation) for some string s – Error: no parse tree for some string s – How do we construct the parse tree? Compiler Design 1 (2011) 2 Abstract Syntax Trees • So far, a parser traces the derivation of a sequence of tokens • The rest of the compiler needs a structural representation of the program • Abstract syntax trees – Like parse trees but ignore some details – Abbreviated as AST Compiler Design 1 (2011) 3 Abstract Syntax Trees (Cont.) • Consider the grammar E → int | ( E ) | E + E • And the string 5 + (2 + 3) • After lexical analysis (a list of tokens) int5 ‘+’ ‘(‘ int2 ‘+’ int3 ‘)’ • During parsing we build a parse tree … Compiler Design 1 (2011) 4 Example of Parse Tree E • Traces the operation of the parser E + E • Captures the nesting structure • But too much info int5 ( E ) – Parentheses – Single-successor nodes + E E int 2 int3 Compiler Design 1 (2011) 5 Example of Abstract Syntax Tree PLUS PLUS 5 2 3 • Also captures the nesting structure • But abstracts from the concrete syntax a more compact and easier to use • An important data structure in a compiler Compiler Design 1 (2011) 6 Semantic Actions • This is what we’ll use to construct ASTs • Each grammar symbol may have attributes – An attribute is a property of a programming language construct
    [Show full text]
  • Backtracking / Branch-And-Bound
    Backtracking / Branch-and-Bound Optimisation problems are problems that have several valid solutions; the challenge is to find an optimal solution. How optimal is defined, depends on the particular problem. Examples of optimisation problems are: Traveling Salesman Problem (TSP). We are given a set of n cities, with the distances between all cities. A traveling salesman, who is currently staying in one of the cities, wants to visit all other cities and then return to his starting point, and he is wondering how to do this. Any tour of all cities would be a valid solution to his problem, but our traveling salesman does not want to waste time: he wants to find a tour that visits all cities and has the smallest possible length of all such tours. So in this case, optimal means: having the smallest possible length. 1-Dimensional Clustering. We are given a sorted list x1; : : : ; xn of n numbers, and an integer k between 1 and n. The problem is to divide the numbers into k subsets of consecutive numbers (clusters) in the best possible way. A valid solution is now a division into k clusters, and an optimal solution is one that has the nicest clusters. We will define this problem more precisely later. Set Partition. We are given a set V of n objects, each having a certain cost, and we want to divide these objects among two people in the fairest possible way. In other words, we are looking for a subdivision of V into two subsets V1 and V2 such that X X cost(v) − cost(v) v2V1 v2V2 is as small as possible.
    [Show full text]
  • Module 5: Backtracking
    Module-5 : Backtracking Contents 1. Backtracking: 3. 0/1Knapsack problem 1.1. General method 3.1. LC Branch and Bound solution 1.2. N-Queens problem 3.2. FIFO Branch and Bound solution 1.3. Sum of subsets problem 4. NP-Complete and NP-Hard problems 1.4. Graph coloring 4.1. Basic concepts 1.5. Hamiltonian cycles 4.2. Non-deterministic algorithms 2. Branch and Bound: 4.3. P, NP, NP-Complete, and NP-Hard 2.1. Assignment Problem, classes 2.2. Travelling Sales Person problem Module 5: Backtracking 1. Backtracking Some problems can be solved, by exhaustive search. The exhaustive-search technique suggests generating all candidate solutions and then identifying the one (or the ones) with a desired property. Backtracking is a more intelligent variation of this approach. The principal idea is to construct solutions one component at a time and evaluate such partially constructed candidates as follows. If a partially constructed solution can be developed further without violating the problem’s constraints, it is done by taking the first remaining legitimate option for the next component. If there is no legitimate option for the next component, no alternatives for any remaining component need to be considered. In this case, the algorithm backtracks to replace the last component of the partially constructed solution with its next option. It is convenient to implement this kind of processing by constructing a tree of choices being made, called the state-space tree. Its root represents an initial state before the search for a solution begins. The nodes of the first level in the tree represent the choices made for the first component of a solution; the nodes of the second level represent the choices for the second component, and soon.
    [Show full text]
  • Compiler Construction
    UNIVERSITY OF CAMBRIDGE Compiler Construction An 18-lecture course Alan Mycroft Computer Laboratory, Cambridge University http://www.cl.cam.ac.uk/users/am/ Lent Term 2007 Compiler Construction 1 Lent Term 2007 Course Plan UNIVERSITY OF CAMBRIDGE Part A : intro/background Part B : a simple compiler for a simple language Part C : implementing harder things Compiler Construction 2 Lent Term 2007 A compiler UNIVERSITY OF CAMBRIDGE A compiler is a program which translates the source form of a program into a semantically equivalent target form. • Traditionally this was machine code or relocatable binary form, but nowadays the target form may be a virtual machine (e.g. JVM) or indeed another language such as C. • Can appear a very hard program to write. • How can one even start? • It’s just like juggling too many balls (picking instructions while determining whether this ‘+’ is part of ‘++’ or whether its right operand is just a variable or an expression ...). Compiler Construction 3 Lent Term 2007 How to even start? UNIVERSITY OF CAMBRIDGE “When finding it hard to juggle 4 balls at once, juggle them each in turn instead ...” character -token -parse -intermediate -target stream stream tree code code syn trans cg lex A multi-pass compiler does one ‘simple’ thing at once and passes its output to the next stage. These are pretty standard stages, and indeed language and (e.g. JVM) system design has co-evolved around them. Compiler Construction 4 Lent Term 2007 Compilers can be big and hard to understand UNIVERSITY OF CAMBRIDGE Compilers can be very large. In 2004 the Gnu Compiler Collection (GCC) was noted to “[consist] of about 2.1 million lines of code and has been in development for over 15 years”.
    [Show full text]
  • Advanced Parsing Techniques
    Advanced Parsing Techniques Announcements ● Written Set 1 graded. ● Hard copies available for pickup right now. ● Electronic submissions: feedback returned later today. Where We Are Where We Are Parsing so Far ● We've explored five deterministic parsing algorithms: ● LL(1) ● LR(0) ● SLR(1) ● LALR(1) ● LR(1) ● These algorithms all have their limitations. ● Can we parse arbitrary context-free grammars? Why Parse Arbitrary Grammars? ● They're easier to write. ● Can leave operator precedence and associativity out of the grammar. ● No worries about shift/reduce or FIRST/FOLLOW conflicts. ● If ambiguous, can filter out invalid trees at the end. ● Generate candidate parse trees, then eliminate them when not needed. ● Practical concern for some languages. ● We need to have C and C++ compilers! Questions for Today ● How do you go about parsing ambiguous grammars efficiently? ● How do you produce all possible parse trees? ● What else can we do with a general parser? The Earley Parser Motivation: The Limits of LR ● LR parsers use shift and reduce actions to reduce the input to the start symbol. ● LR parsers cannot deterministically handle shift/reduce or reduce/reduce conflicts. ● However, they can nondeterministically handle these conflicts by guessing which option to choose. ● What if we try all options and see if any of them work? The Earley Parser ● Maintain a collection of Earley items, which are LR(0) items annotated with a start position. ● The item A → α·ω @n means we are working on recognizing A → αω, have seen α, and the start position of the item was the nth token. ● Using techniques similar to LR parsing, try to scan across the input creating these items.
    [Show full text]
  • Lecture 3: Recursive Descent Limitations, Precedence Climbing
    Lecture 3: Recursive descent limitations, precedence climbing David Hovemeyer September 9, 2020 601.428/628 Compilers and Interpreters Today I Limitations of recursive descent I Precedence climbing I Abstract syntax trees I Supporting parenthesized expressions Before we begin... Assume a context-free struct Node *Parser::parse_A() { grammar has the struct Node *next_tok = lexer_peek(m_lexer); following productions on if (!next_tok) { the nonterminal A: error("Unexpected end of input"); } A → b C A → d E struct Node *a = node_build0(NODE_A); int tag = node_get_tag(next_tok); (A, C, E are if (tag == TOK_b) { nonterminals; b, d are node_add_kid(a, expect(TOK_b)); node_add_kid(a, parse_C()); terminals) } else if (tag == TOK_d) { What is the problem node_add_kid(a, expect(TOK_d)); node_add_kid(a, parse_E()); with the parse function } shown on the right? return a; } Limitations of recursive descent Recall: a better infix expression grammar Grammar (start symbol is A): A → i = A T → T*F A → E T → T/F E → E + T T → F E → E-T F → i E → T F → n Precedence levels: Nonterminal Precedence Meaning Operators Associativity A lowest Assignment = right E Expression + - left T Term * / left F highest Factor No Parsing infix expressions Can we write a recursive descent parser for infix expressions using this grammar? Parsing infix expressions Can we write a recursive descent parser for infix expressions using this grammar? No Left recursion Left-associative operators want to have left-recursive productions, but recursive descent parsers can’t handle left recursion
    [Show full text]