Top Down Parsing Example Example #2 Solutions?

Home , Backtracking, Parse tree

Top Down Parsing Example Suppose we have a grammar: A legal parse would be: . When we are parsing, we produce a unique syntax tree from a legal 1. S 1 + 2 * 3 sentence. S → E only one rule: S → E —An unambiguous grammar gives rise to a single leftmost E → T | E+T 2. E 1 + 2 * 3 derivation for any sentence in the language. T → F | T*F choose E → E+T . So, if we are trying to recognise a sentence, what we are trying to do F → unit | (E) 3. E+T 1 + 2 * 3 is grow a parse tree corresponding to that sentence choose E → T → F→ unit —We are trying to find the leftmost derivation. and the expression: 4. unit+T 1 + 2 * 3 . A top-down parser constructs a leftmost parse 1 + 2 * 3 match 1 and + —We will always be looking at the leftmost nonterminal. 5. 1+T 2 * 3 This means that to choose T → F→ unit . This follows the push down automaton model of the previous lecture parse this sentence 6. 1+unit 2 * 3 . The parser must choose the correct production from the set of some backtracking is match 2 productions that correspond to the current state of the parse. required, i.e., put input 7. 1+2 *3 . If at any time there is no candidate production corresponding to the symbols back! ✸WRONG state of the parse, we must have made a wrong turn at some earlier Backtracking in stage and we will need to backtrack. compilers is nontrivial and to be avoided!!

Example #2 Solutions? Suppose we have a grammar: A legal parse would be: . We could rearrange the productions so that the left recursive ones 1. S 1 + 2 * 3 come at the end, and always choose the first matching production. S → E only one rule: S → E . For the previous examples, this has already been done. The left E → T | E+T 2. E 1 + 2 * 3 recursive ones are at the end of the list! T → F | T*F choose E → E+T . Note that this is not an easy task in general since mutually F → unit | (E) 3. E+T 1 + 2 * 3 recursive grammars have the same problems: choose E → E+T and the expression: 4. E+T+T 1 + 2 * 3 A → B | C D 1 + 2 * 3 choose E → E+T C → E | A F 5. E+T+T+T 1 + 2 * 3 . In general, rearranging productions will not help – the parser will choose E → E+T still have problems. ...you can see what happens! —Even if it does help, a parser which needs to backtrack an arbitrary distance is inefficient. The problem is simple: Left Recursion! . What we need is a way to deterministically parse a grammar in a top down fashion without backtracking.

1 Eliminating left recursion Example of eliminating left recursion . An algorithm to eliminate arbitrary left recursion (by replacing it . Consider the productions: with right recursion) is as follows: A → a | Ba B → b | Cb C → c | Ac

1. Arbitrarily order the non-terminals: N1, N2, N3, ... 1. Arbitrarily order the non-terminals: A, B, C

2. Apply the following steps to the productions for N1, then N2, ... 2. Consider the productions for A: no change

3. For Ni: 2. Consider the productions for B: no change

a) For all productions Ni → Nk α, where k < i and if the productions 3. Consider the productions for C: for Nk are Nk → β1 | β2 | β3 | ... then expand the reference to Nk, a) Replace C → Ac by C → ac | Bac i.e. replace the production N N by N ... i → k α i → β1 α | β2 α | a) Replace C → Bac by C → bac | Cbac b) If the productions for N are now i Productions for C are now: C → c | ac | bac | Cbac Ni → α1 | α2 | ... | Ni β1 | Ni β2 | ... (where the first few are not left recursive while the latter are) b) Replace the productions for C by: then replace them with C → cC’ | acC’ | bacC’ Ni → α1 Ni’ | α2 Ni’ | ... C’ → ε | bacC’ Ni’ → ε | β1 Ni’ | β2 Ni’ | ...

A Workable Solution Definitions Observation . A parser which can make a deterministic decision about which . The trouble which gives rise to nondeterminacy and alternative to choose when faced with one, if given a buffer of k backtracking in top down parsers shows itself in only one symbols, is called a LL(k) parser. place – that is when a parser has to choose between —Left to right scan of input several alternatives with the same left hand side. —Left most derivation . The only information which we can use to make the —k symbols of look-ahead correct decision is the input stream itself. . The grammar that an LL(k) parser recognizes is an LL(k) grammar and any language that has an LL(k) grammar is an LL(k) language. —In the example, we (humans) could see which alternative to choose by looking at the input yet-to-be-read. —We are constructing an LL(1) compiler that recognises LL(1) grammars. . If we are going to look ahead in order to make the correct —So the question is How do we know when we have an LL(1) decision, we need a buffer in which to store the next few grammar? symbols. . We also have LR(k) grammars and other variations, but our focus is . In practice, this buffer is of a fixed length. currently on LL(1) grammars.

2 Definition of LL(1) Definition of First . When faced with a production such as: . To compute FIRST(X) for all grammar symbols X, apply the following algorithm until no more terminals or can be A → α | α | α ε 1 2 3 added to any FIRST set. . We chose one of the α uniquely by looking at the i 1. If X is a terminal, then FIRST(X) is {X} next input symbol. 2. If X → ε is a production, then add ε to FIRST(X) . We employ two sets: first and follow, to help us. 3. If X is a nonterminal and X → Y1 Y2 ...Yn is a Recall: production, then place a in FIRST(X) if for some i, a is in

. First(X) is the set of all terminal symbols that can FIRST(Yi), and ε is in all of FIRST(Y1 ), ..., FIRST (Yi-1 ); “start” the production X that is Y1 Y2 ...Yi-1 ⇒ * ε . If ε is in FIRST(Yj ) for all j = 1, 2, ... n, then add ε to . Follow(X) is the set of terminal symbols that can FIRST(X). For example, everything in FIRST(Y ) is surely follow an “X” 1 in FIRST(X). If Y1 does not derive ε, then we add nothing more to FIRST(X), but if Y1 ⇒ ε then we add FIRST(Y2) and so on.

Definition of Follow Definition of LL(1) property . To compute FOLLOW(A) for all nonterminals A, apply the following algorithm until nothing can be added to any FOLLOW set. 1. Place $ in FOLLOW(S), where S is the start symbol and Definition: A grammar G is LL(1) if and only if for all rules $ is the input right endmarker. A → α1 | α2 | ... | αn

2. If there is a production A → α B β then everything in . director(αi) ∩ director(αk) = ∅ ∀ i ≠ k FIRST(β) except for ε is placed in FOLLOW(B). where: ∗ director( ) = first( ) follow(A) if 3. If there is a production A → αB or a production A → α Bβ αi αi ∪ αj ⇒ ε = first(α ) otherwise where FIRST(β) contains ε (i.e., β ⇒ * ε), then i everything in FOLLOW(A) is in FOLLOW(B).