Part III: Context-Free Languages and Pushdown Automata In this section, we move out one level and explore the class of context-free languages. This class is important. For most programming languages, the set of syntactically legal statements is (except possibly for type checking) a context-free language. The set of well-formed Boolean queries is a context-free language. A great deal of the syntax of English can be described in the context-free framework that we are about to discuss. To describe these languages, we need more power than the regular language definition allows. For example, to describe both programming language statements and Boolean queries requires the ability to specify that parentheses be balanced. Yet we showed in Section 8.4 that it is not possible to define a regular language that contains exactly the set of strings of balanced parentheses. We will begin our discussion of the context-free languages by defining a grammatical formalism that can be used to describe every language in the class (which, by the way, does include the language of balanced parentheses). Then, in Chapter 12, we will return to the question of defining machines that can accept strings in the language. At that point, we’ll see that the pushdown automaton, an NDFSM augmented with a single stack, can accept exactly the class of context-free languages that we are about to describe. In Chapter 13, we will see that the formalisms that we have presented stop short of the full power that is provided by a more general computational model. So we’ll see that there are straightforward languages that are not context-free. But, because of the restrictions that the context-free formalism imposes, it will turn out to be possible to define algorithms that perform at least the most basic operations on context- free languages, including deciding whether a string is in a language. We’ll summarize those algorithms in Chapters 14 and 15. The theory that we are about to present for the context-free languages is not as straightforward and elegant as the one that we have just described for the regular languages. We’ll see, for example, that there doesn’t exist an algorithm that compares two pushdown automata to see if they are equivalent. Given an arbitrary context-free grammar G, there doesn’t exist a linear-time algorithm that decides whether a string w is an element of L(G). But there does exist such an algorithm if we restrict our attention to a useful subset of the context-free languages. The context-free languages are not closed under many common operations like intersection and complement. On the other hand, because the class of context-free languages includes most programming languages, query languages, and a host of other languages that we use daily to communicate with computers, it is worth taking the time to work through the theory that is presented here, even though it is less clear than the one we were able to build in Part II. Part III 145 Context-Free Grammars and Pushdown AutomataContext-Free Grammars 11 Context-Free Grammars We saw, in our discussion of the regular languages in Part II, that there are substantial advantages to using descriptive frameworks (in that case, FSMs, regular expressions, and regular grammars) that offer less power and flexibility than a general purpose programming language provides. Because the frameworks were restrictive, we were able to describe a large class of useful operations that could be performed on the languages that we defined. We will begin our discussion of the context-free languages with another restricted formalism, the context-free grammar. But before we define it, we will pause and answer the more general question, “What is a grammar?” 11.1 Introduction to Rewrite Systems and Grammars We’ll begin with a very general computational model: Define a rewrite system (also called a production system or a rule-based system) to be a list of rules and an algorithm for applying them. Each rule has a left-hand side and a right- hand side. For example, the following could be rewrite-system rules: S → aSb aS → aSb → bSabSa In the discussion that follows, we will focus on rewrite system that operate on strings. But the core ideas that we will present can be used to define rewrite systems that operate on richer data structures. Of course, such data structures can be represented as strings, but the power of many practical rule-based systems comes from their ability to manipulate other structures directly. Expert systems, C 774, are programs that perform tasks in domains like engineering, medicine, and business, that require expertise when done by people. Many kinds of expertise can naturally be modeled as sets of condition/action rules. So many expert systems are built using tools that support rule-based programming. Rule based systems are also used to model business practices, C 774, and as the basis for reasoning about the behavior of nonplayer characters in computer games, C 791. When a rewrite system R is invoked on some initial string w, it operates as follows: simple-rewrite(R: rewrite system, w: initial string) = 1. Set working-string to w. 2. Until told by R to halt do: 2.1. Match the left-hand side of some rule against some part of working-string. 2.2. Replace the matched part of working-string with the right-hand side of the rule that was matched. 3. Return working-string. If simple-rewrite(R, w) can return some string s then we’ll say that R can derive s from w or that there exists a derivation in R of s from w. Rewrite systems can model natural growth processes, as occur, for example, in plants. In addition, evolutionary algorithms can be applied to rule sets. Thus rewrite systems can model evolutionary processes, C 809. We can define a particular rewrite-system formalism by specifying the form of the rules that are allowed and the algorithm by which they will be applied. In most of the rewrite-system formalisms that we will consider, a rule is simply a pair of strings. If the string on the left-hand side matches, it is replaced by the string on the right-hand side. Chapter 11 146 Context-Free Grammars But more flexible forms are also possible. For example, variables may be allowed. Let x be a variable. Then consider the rule: axa → aa This rule will squeeze out whatever comes between a pair of a’s. Another useful form allows regular expressions as left-hand sides. If we do that, we can write rules like the following, which squeezes out b’s between a’s: ab*ab*a → aaa The extended form of regular expressions that is supported in programming languages like Perl is often used to write substitution rules. C 792. In addition to describing the form of its rules, a rewrite-system formalism must describe how its rules will be applied. In particular, a rewrite-system formalism will define the conditions under which simple-rewrite will halt and the method by which it will choose a match in step 2.1. For example, one rewrite-system formalism might specify that any rule that matches may be chosen. A different formalism might specify that the rules have to be tried in the order in which they are written, with the first one that matches being the one that is chosen next. Rewrite systems can be used to define functions. In this case, we write rules that operate on an input string to produce the required output string. And rewrite systems can be used to define languages. In this case, we define a unique start symbol. The rules then apply and we will say that the language L that is generated by the system is exactly the set of strings, over L’s alphabet, that can be derived by simple-rewrite from the start symbol. A rewrite-system formalism can be viewed as a programming language and some such languages turn out to be useful. For example, Prolog, C 762, supports a style of programming called logic programming. A logic program is a set of rules that correspond to logical statements of the form A if B. The interpreter for a logic program reasons backwards from a goal (such as A), chaining rules together until each right-hand side has been reduced to a set of facts (axioms) that are already known to be true. The study of rewrite systems has played an important role in the development of the theory of computability. We’ll see in Part V that there exist rewrite-system formalisms that have the same computational power as the Turing machine, both with respect to computing functions and with respect to defining languages. In the rest of our discussion in this chapter, however, we will focus just on their use to define languages. A rewrite system that is used to define a language is called a grammar. If G is a grammar, let L(G) be the language that G generates. Like every rewrite system, every grammar contains a list (almost always treated as a set, i.e., as an unordered list) of rules. Also, like every rewrite system, every grammar works with an alphabet, which we can call V. In the case of grammars, we will divide V into two subsets: • a terminal alphabet, generally called , which contains the symbols that make up the strings in L(G), and • a nonterminal alphabet, the elements of which will function as working symbols that will be used while the grammar is operating.