Context-free Grammar Problems with Regular Context-free Grammar Language and Is English a regular language? Bad question! We do not even know what English is! Two eggs and bacon make(s) a big breakfast Backtrack Parsing Can you slide me the salt? He didn't ought to do that But—No! Martin Kay I put the wine you brought in the fridge I put the wine you brought for Sandy in the fridge Should we bring the wine you put in the fridge out Stanford University now? and University of the Saarland You said you thought nobody had the right to claim that they were above the law Martin Kay Context-free Grammar 1 Martin Kay Context-free Grammar 2 Problems with Regular Problems with Regular Language Language You said you thought nobody had the right to claim [You said you thought [nobody had the right [to claim that they were above the law that [they were above the law]]]] Martin Kay Context-free Grammar 3 Martin Kay Context-free Grammar 4 Problems with Regular Context-free Grammar Language Nonterminal symbols ~ grammatical categories Is English mophology a regular language? Bad question! We do not even know what English Terminal Symbols ~ words morphology is! They sell collectables of all sorts Productions ~ (unordered) (rewriting) rules This concerns unredecontaminatability Distinguished Symbol This really is an untiable knot. But—Probably! (Not sure about Swahili, though) Not all that important • Terminals and nonterminals are disjoint • Distinguished symbol Martin Kay Context-free Grammar 5 Martin Kay Context-free Grammar 6 Context-free Grammar Context-free Grammar Nonterminal symbols ~ grammatical categories Nonterminal symbols ~ grammatical categories S, NP, VP, Det, N, V S, NP, VP, Det, N, V Terminal Symbols ~ words Terminal Symbols ~ words the, dog, cat, chased the, dog, cat, chased Productions ~ (unordered) (rewriting) rules Productions ~ (unordered) (rewriting) rules S → NP VP Det → the S → NP VP Det → the NP → Det N N → dog NP → Det N N → dog VP → V NP N → cat VP → V NP N → cat V → chased V → chased Distinguished Symbol Distinguished Symbol S S Martin Kay Context-free Grammar 7 Martin Kay Context-free Grammar 8 Context-free Grammar Context-free Grammar Nonterminal symbols ~ grammatical categories Nonterminal symbols ~ grammatical categories S, NP, VP, Det, N, V S, NP, VP, Det, N, V Terminal Symbols ~ words Terminal Symbols ~ words the, dog, cat, chased the, dog, cat, chased Productions ~ (unordered) (rewriting) rules Productions ~ (unordered) (rewriting) rules S → NP VP Det → the S → NP VP Det → the S S NP → Det N N → dog NP → Det N N → dog NP VP NP VP VP → V NP N → cat VP → V NP N → cat V → chased V → chased Det N Distinguished Symbol Distinguished Symbol S S Martin Kay Context-free Grammar 9 Martin Kay Context-free Grammar 10 Context-free Grammar Context-free Grammar Nonterminal symbols ~ grammatical categories Nonterminal symbols ~ grammatical categories S, NP, VP, Det, N, V S, NP, VP, Det, N, V Terminal Symbols ~ words Terminal Symbols ~ words the, dog, cat, chased the, dog, cat, chased Productions ~ (unordered) (rewriting) rules Productions ~ (unordered) (rewriting) rules S → NP VP Det → the S → NP VP Det → the S S NP → Det N N → dog NP → Det N N → dog NP VP NP VP VP → V NP N → cat VP → V NP N → cat V → chased Det N V NP V → chased Det N V NP Distinguished Symbol Distinguished Symbol Det N S S Martin Kay Context-free Grammar 11 Martin Kay Context-free Grammar 12 Context-free Grammar Context-free Grammar Nonterminal symbols ~ grammatical categories Nonterminal symbols ~ grammatical categories S, NP, VP, Det, N, V S, NP, VP, Det, N, V Terminal Symbols ~ words Terminal Symbols ~ words the, dog, cat, chased the, dog, cat, chased Productions ~ (unordered) (rewriting) rules Productions ~ (unordered) (rewriting) rules S → NP VP Det → the S → NP VP Det → the S S NP → Det N N → dog NP → Det N N → dog NP VP NP VP VP → V NP N → cat VP → V NP N → cat V → chased Det N V NP V → chased Det N V NP Distinguished Symbol the Det N Distinguished Symbol the dog chasedDet N S S the cat Martin Kay Context-free Grammar 13 Martin Kay Context-free Grammar 14 Context-free Grammar Context-free Grammar Nonterminal symbols ~ grammatical categories S, NP, VP, Det, N, V Terminal Symbols ~ words the, dog, cat, chased • Defines a language (set of strings) and Productions ~ (unordered) (rewriting) rules S → NP VP Det → the • Defines a corresponding set of structures S NP → Det N N → dog NP VP VP → V NP N → cat V → chased Det N V NP Distinguished Symbol the dog chasedDet N S the cat Martin Kay Context-free Grammar 15 Martin Kay Context-free Grammar 16 Structure and Semantics Aithmetic Expressions sum ! prod add_op sum sum sum ! prod 14 sum prod ! base mul_op prod prod ad_op sum prod prod ! base 12 base + prod ad_op base base ! a | b | c ... ( sum ) base - y num plus num times num base ! ( sum ) 2 3 4 ad_op ! + prod ad_op sum x ad_op ! - 2 + 3 ! 4 base + prod mul_op ! * a base mul_op ! / b Martin Kay Context-free Grammar 17 Martin Kay Context-free Grammar 18 Aithmetic Expressions Embedding sum sum ! prod add_op sum Arbitrarily embedded parentheses are not prod sum ! prod possible in a regular language because the number of open parentheses that still prod ! base mul_op prod base requiring matching close parentheses can be prod ! base ( sum ) greater than the number of states in the base ! a | b | c ... automaton, whatever that number is. prod base ! ( sum ) Hence, by the pumping lemma, an automaton ad_op ! + base with only a finite number of states cannot characterize the language. ad_op ! - ( sum ) mul_op ! * . mul_op ! / Martin Kay Context-free Grammar 19 Martin Kay Context-free Grammar 20 Ambiguity Lots of Problems! More than one structure can correspond to a That it rained slept That it rained were single string soundly. certainly true. S Mary told the boy Mary His arguments convinces me. John told that it rained NP VP me Me told Mary it. Det N Vi That can burst For the moment, just forget the problems Pro aux Vi NP VP S Martin Kay Context-free Grammar 21 Martin Kay Context-free Grammar 22 Why "Context-free"? Convenience You can define something once, and then use it in many places in the grammar S S S NP VP NP VP Det N V NP NP the dog chased Det N Vd NP NP S the cat S NP VP VP NP VP What can be in this subtree Vi Vt NP is determined only by this symbol Vi and not by any of this "context" John told me that it rained That it rained surprised John Martin Kay Context-free Grammar 23 Martin Kay Context-free Grammar 24 How many (binary) trees How many (binary) trees with n terminals? with n terminals? Any binary number Any binary number represents a set of represents a set of binary branching trees binary branching trees 2n-2 2n-2 Exponential Exponential ⬈ ⬈ ⬉ ⬉ ⬈ ⬈ ⬉ ⬈ ⬉ ⬉ ⬈ ⬈ ⬉ ⬈ ⬈ ⬉ ⬉ ⬈ ⬈ ⬉ ⬈ ⬉ ⬉ ⬈ ⬈ ⬉ 1 1 1 0 0 1 1 0 1 0 0 1 1 0 1 1 1 0 0 1 1 0 1 0 0 1 1 0 Martin Kay Context-free Grammar 25 Martin Kay Context-free Grammar 26 Catalan Numbers Catalan Numbers 1 terminal 2 terminals Terminals Trees Terminals Trees 1 1 1 1 2 1 Why? Martin Kay Context-free Grammar 27 Martin Kay Context-free Grammar 28 Catalan Numbers Catalan Numbers 3 terminals 3 terminals Terminals Trees Terminals Trees left ! 1 1 1 1 ! right 2 1 ! right left ! 2 1 Martin Kay Context-free Grammar 29 Martin Kay Context-free Grammar 30 Catalan Numbers Catalan Numbers 4 terminals 4 terminals Terminals Trees Terminals Trees left ! 1 1 1 1 2 1 left ! 2 1 ! right 3 2 ! right 3 2 1 ! 2 = 2 1 ! 2 = 2 1 ! 1 = 1 Martin Kay Context-free Grammar 31 Martin Kay Context-free Grammar 32 Catalan Numbers Catalan Numbers 4 terminals 4 terminals Terminals Trees Terminals Trees 1 1 ! right 1 1 ! right 2 1 2 1 left ! 3 2 left ! 3 2 4 5 1 ! 2 = 2 1 ! 2 = 2 1 ! 1 = 1 1 ! 1 = 1 2 ! 1 = 2 2 ! 1 = 2 5 Martin Kay Context-free Grammar 33 Martin Kay Context-free Grammar 34 Catalan Numbers Catalan Numbers 5 terminals 4 terminals Terminals Trees Terminals Trees left ! 1 1 1 1 2 1 left ! 2 1 3 2 3 2 ! right 4 5 ! right 4 5 1 ! 5 = 5 1 ! 5 = 5 1 ! 2 = 2 Martin Kay Context-free Grammar 35 Martin Kay Context-free Grammar 36 Catalan Numbers Catalan Numbers 4 terminals 4 terminals Terminals Trees Terminals Trees 1 1 1 1 ! right 2 1 ! right 2 1 left ! 3 2 3 2 4 5 left ! 4 5 1 ! 5 = 5 1 ! 5 = 5 1 ! 2 = 2 1 ! 2 = 2 2 ! 1 = 2 2 ! 1 = 2 5 ! 1 = 5 14 Martin Kay Context-free Grammar 37 Martin Kay Context-free Grammar 38 Catalan Numbers Catalan Numbers The number of binary trees with n terminals Terminals Trees 1 1 The number of arbitrarily branching trees 2 1 with n nodes. 3 2 (= 1*1 + 1*1) 4 5 (= 1*2 + 1*1 + 2*1) 5 14 (= 1*5 + 1*2 + 2*1 + 5*1) 6 42 (= 1*14 + 1*5 + 2*2 + 5*1 + 14*1) ... Any tree can be encoded in a 2n - 1 binary tree with the same Cat(n) = n number of terminals as the 2n - 1 original tree has nodes. Martin Kay Context-free Grammar 39 Martin Kay Context-free Grammar 40 Catalan Numbers Analyzing Sentences There are n nonterminals in the binary tree for the n nodes in the original tree, except for the root.