A FORMAL MODEL FOR CONTEXT-FREE LANGUAGES AUGMENTED WITH REDUPLICATION
Walter J. Savitch Department of Computer Science and Engineering University of California, San Diego La Jolla, CA 92093
A model is presented to characterize the class of languages obtained by adding reduplication to context-free languages. The model is a pushdown automaton augmented with the ability to check reduplication by using the stack in a new way. The class of languages generated is shown to lie strictly between the context-free languages and the indexed languages. The model appears capable of accommodating the sort of reduplications that have been observed to occur in natural languages, but it excludes many of the unnatural constructions that other formal models have permitted.
1 INTRODUCTION formal language. Indeed, most of the known, convinc- ing arguments that some natural language cannot (or Context-free grammars are a recurrent theme in many, almost cannot) be weakly generated by a context-free perhaps most, models of natural language syntax. It is grammar depend on a reduplication similar to the one close to impossible to find a textbook or journal that exhibited by this formal language. Examples include the deals with natural language syntax but that does not respectively construct in English (Bar-Hillel and Shamir include parse trees someplace in its pages. The models 1964), noun-stem reduplication and incorporation in used typically augment the context-free grammar with Mohawk (Postal 1964), noun reduplication in Bambara some additional computational power so that the class (Culy 1985), cross-serial dependency of verbs and ob- of languages described is invariably larger, and often jects :in certain subordinate clause constructions in much larger, than the class of context-free languages. Dutch (Huybregts 1976; Bresnan et al. 1982) and in Despite the tendency to use more powerful models, Swiss-German (Shieber 1985), and various reduplica- examples of natural language constructions that require tion constructs in English, including the X or no X more than a context-free grammar for weak generative construction as in: "reduplication or no reduplication, I adequacy are rare. (See Pullum and Gazdar 1982, and want a parse tree" (Manaster-Ramer 1983, 1986). The Gazdar and Pullum 1985, for surveys of how rare model presented here can generate languages with any inherently noncontext-free constructions appear to be.) of these constructions and can do so in a natural way. Moreover, most of the examples of inherently noncon- To have some concrete examples at hand, we will text-free constructions in natural languages depend on a review a representative sample of these constructions. single phenomenon, namely the reduplication, or ap- The easiest example to describe is the noun redupli- proximate reduplication, of some string. Reduplication cation found in Bambara. As described by Culy (1985), is provably beyond the reach of context-free grammars. it is an example of the simplest sort of reduplication in The goal of this paper is to present a model that can which a string literally occurs twice. From the noun w accommodate these reduplication constructions with a Bambara can form w-o-w with the meaning "whatever minimal extension to the context-free grammar model. w." It is also possible to combine nouns productively in Reduplication in its cleanest, and most sterile, form other ways to obtain new, longer nouns. Using these is represented by the formal language {ww I w ~ E*}, longer nouns in the w-o-w construction produces redu- where E is some finite alphabet. It is well known that plicated strings of arbitrary length. this language is provably not context-free. Yet there are The respectively construction in English is one of the numerous constructs in natural language that mimic this oldest well-known examples of reduplication (Bar-Hillel
Copyright 1989 by the Association for Computational Linguistics. Permission to copy without fee all or part of this material is granted provided that the copies are not made for direct commercial advantage and the CL reference and this copyright notice are included on the first page. To copy otherwise, or to republish, requires a fee and/or specific permission. 0362-613 X/89/010250-261 $ 03.00
250 ComputationaH Linguistics, Volume 15, Number 4, December 1989 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication
and Shamir 1964). It provides an example of reduplica- can be weakly generated by a context-free grammar, tion other than exact identity. A sample sentence is: even though it cannot be strongly generated by a John, Sally, Mary, and Frank are a context-free grammar. The context-free grammar to widower, widow, widow, and widower, respectively. generate the strings would pair nouns and verbs in a In these cases, it has been argued that the names must mirror image manner, thereby ensuring that there are agree with "widow" or "widower" in gender, and equal numbers of each. Since Dutch does not have the hence the string from {widow, widower}* must be an strong agreement rule that Swiss-German does, this approximate reduplication of the string of names. If one always produces a grammatical clause, even though the accepts the data, then this is an example of reduplica- pairing of nouns and verbs is contrary to intuition. tion using a notion of equivalence other than exact However, in cases such as this, it would be desirable to identity. In this case the equivalence would be that the have a model that recognizes reduplication as redupli- second string is a homomorphic image of the first one. cation rather than one that must resort to some sort of However, one must reject the data in this case. One can trick to generate weakly the reduplicated strings in a convincingly argue that the names need not agree with highly unnatural manner. This is true even if one is the corresponding occurrence of "widow" or "widow- seeking only weak generative capacity because, as the er," because gender is not syntactic in this case. It may Dutch/Swiss-German pair indicates, if a minor and be false, but it is not ungrammatical to say "John is a plausible addition to a construct in one natural language widow." (Perhaps it is not even false, since no syntactic would make it demonstrably noncontext-free, then we rule prevents parents from naming a daughter "John.") can suspect that some other language may exhibit this However, such dependency is at least potentially pos- or a similar inherently noncontext-free property, even sible in some language with truly syntactic gender when considered purely as a string set. markings. Kac et al. (1987) discuss a version of this Some of these arguments are widely accepted. Oth- construction using subject-verb number agreement that ers are often disputed. We will not pass judgment here yields a convincing argument that English is not a except to note that, whether or not the details of the context-free language. data are sharp enough to support a rigorous proof of One of the least controversial arguments claiming to noncontext-freeness, it is nonetheless clearly true that, prove that a particular natural language cannot be in all these cases, something like reduplication is occur- weakly generated by a context-free grammar is Shie- ring. A model that could economically capture these ber's (1985) argument about Swiss-German. In this constructions as well as any reasonable variant on these case, the reduplication occurs in certain subordinate examples would go a long way toward the goal of clauses such as the following: precisely describing the class of string sets that corre- ... mer em Hans es huus h/ilfed aastriiche spond to actual and potential human languages. ... we Hans-DAT the house-ACC helped paint We do not contend that the model to be presented '....we helped Hans paint the house.' here will weakly describe all natural languages without where to obtain a complete sentence, the above should even the smallest exception. Any such claim for any be preceded by some string such as "Jan s/iit das" ('Jan model, short of ridiculously powerful models, is says that'). In this case, a list of nouns precedes a list of doomed to failure. Human beings taken in their entirety an equal number of verbs and each noun in the list are general-purpose computers capable of performing serves as the object of the corresponding verb. The any task that a Turing machine or other general-purpose cross-serial dependency that pushes the language be- computer model can perform, and so humans can yond the reach of a context-free grammar is an agree- potentially recognize any language describable by any ment rule that says that each verb arbitrarily demands algorithmic process whatsoever (although sometimes either accusative ordative case for its object. Thus if we too slowly to be of any practical concern). The human substitute "de Hans" (Hans-ACC) for "em Hans" language facility appears to be restricted to a much less (Hans-DAT) or "em huus" (the house-DAT) for "es powerful computational mechanism. However, since huus" (the house-ACC), then the above is ungrammat- the additional power is there for purposes other than ical because "h/ilfed" demands that its object be in the language processing, some of this power inevitably will dative case and "aastriiche" requires the accusative find its way into language processing in some small case. Since the lists of nouns and verbs may be of measure. Indeed, we discuss one Dutch construction unbounded length, this means that Swiss-German con- that our model cannot handle. We claim that our model tains substrings of the forms captures most of the known constructions that make N 1 N2"'Nn VI V2... Vn natural language provably not context-free as string where n may be arbitrarily large and where each noun sets, and that it does so with a small addition to the N i is in either the dative or accusative case depending context-free grammar model. No more grandiose claims on an arbitrary requirement of the verb Vi. are made. Bresnan et al. (1982) describe a similar construction It is easy to add power to a model, and there are in Dutch in which the strong agreement rule is not numerous models that can weakly generate languages present and so the language (at least this aspect of it) representing all of these noncontext-free constructions.
Computational Linguistics, Volume 15, Number 4, December 1989 251 Walter J. Savitch A Formal Modeli for Context-Free Languages Augmented with Reduplication
However, they all appear to be much too powerful for noted ~, and a special type of instruction to check for the simple problems that extend natural language be- reduplication. The symbol $ is inserted in the stack just yond the capacity of context-free grammar. One of the like any other stack symbol, and the stack grows above less powerful of the well-known models is indexed this symbol just as in an ordinary PDA. To check for an grammar, as introduced by Aho (1968) and more re- occurrence of a simple reduplication ww, the RPDA cently summarized in the context of natural language by pushe,; $ onto the stack and then pushes the first w onto Gazdar (1985). However, even the indexed languages the stack symbol by symbol. At that point the stack appear to be much more powerful than is needed for contains + with w on top of it, but while the stack natural language syntax. We present a model that is symbols are ordered so that it would be easy to compare weaker than the indexed grammar model, simpler than the w in the stack with w n (i.e., w reversed), they are in the indexed grammar model, and yet capable of han- the wrong order to compare them with w. To overcome dling all context-free constructs plus reduplication. this; problem an RPDA is allowed, in one step, to A number of other models extend the context-free corapare the entire string above the $ to an initial grammar model in a limited way. Four models that are segment of the remaining input and to do so in the order known to be weakly equivalent and to be strictly weaker starting at the special symbol ~ rather than at the top of than indexed grammars are: the Tree Adjoining Gram- the stack. (The comparison consumes both the stack mars (TAGs) of Joshi (1985, 1987), the Head Grammars contents above the marker ~ and the input with which it of Pollard (1984), the Linear Indexed Grammars of is compared.) One way to view this is to say that the Gazdar (1985), and the Combinatory Categorial Gram- RPDA can decide to treat the stack above the symbol mars of Steedman (1987, 1988). For a discussion of this like a queue, but once it decides to do so, all that it equivalence see Joshi et al. (1989). The oldest of these can do is empty the queue. The RPDA cannot add to the four models is the TAG grammar of Joshi, and we shall queue once it starts to empty it. If the RPDA decides to refer to the class of languages generated by any of these check xy to see if x = y, and x does not in fact equal y equivalent grammar formalisms as TAG languages. (or any initial segment of y), then the computation However, the reader should keep in mind that this class block,;. While placing symbols on top of the symbol $, of languages could be represented by any of the four the stack may grow or shrink just like the stack on an equivalent grammar formalisms. As we will see later in ordinary PDA. The model allows more than one marker this paper, there are TAG languages that cannot be $ to be placed in the stack, and hence, it can check for weakly generated by our model. Our model seems to redup!lications nested within reduplications. exclude more unnatural strings sets than these models Because an RPDA is free to push something on the do. Of course, our model may also miss some natural stack other than w when processing some input wx, the string sets that are TAG languages. Recent work of model can check not only whether w = x, but also can Joshi (1989) appears to support our conjecture that the check the more general question of whether some class of language described by our model is a subset, specific finite-state transduction maps w onto x. A and hence a strict subset, of the TAG languages. finJite-state transduction is any relation that can be However, all the details of the proof have not yet been computed using only a finite-state machine. The finite- worked out, and so any more detailed comparisons to state transductions include all of the simple word-to- TAG languages will be left for another paper. This paper assumes some familarity with the notation word equivalences used in known reduplication and results of formal language theory. Any reader who constructions. For example, for the Swiss-German sub- has worked with context-free grammars, who knows ordinate clauses described by Shieber, w is a string of what a pushdown automaton (PDA) is, and who knows nouns, marked for either dative or accusative case, and what it means to say that PDAs accept exactly the x is a string of verbs that each select one and only one context-free languages should have sufficient back- of the: cases for their corresponding, cross-serially lo- ground to read this paper. Any needed background can cated object noun. The finite-state transduction would be found in almost any text on formal language theory, map each noun in the accusative case onto a verb such as Harrison (1978) or Hopcroft and Ullman (1979). nondeterministically chosen from the finite set of verbs that take the accusative case, and would do a similar thing with nouns in the dative case and their corre- 2 THE RPDA MODEL sponding class of verbs. Without this, or some similar The model we propose here is an automata-based model generality, the only reduplication allowed would be similar to the pushdown automaton that characterizes exact symbols by symbol identity. the context-free languages. A formal definition will The formal details of the definitions are now routine, follow shortly, but the informal description given now but to avoid any misunderstanding, we present them in should be understandable to anybody who has worked some detail. in this area. The model is called a Reduplication PDA, or 1. A Reduplication PDA (abbreviated RPDA) consists more simply an RPDA. It consists of an ordinary PDA of the following items: augmented with a special stack symbol, which is de- (i) A finite set of states S, an element qo in S to serve
252 Computational Linguistics, Volume 15, Number 4, December 1989 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication
as the start state, and a finite subset F of S to serve as deterministic RPDA by final state, then it is called a the accepting states; deterministic RPDA language. (ii) A finite input alphabet E; 6. The language accepted by the RPDA M by empty (iii) A finite pushdown store alphabet Fsuch that E C store is defined and denoted as follows: N(M) = { w I (qo, F, a distinguished symbol Z o in F to serve as the start w, Z ) F* (p, A, A) for some p ~ S}. pushdown symbol, and a distinguished stack marker ~, As in the case of ordinary PDAs, it turns out that for which is an element of F-E; RPDAs acceptance by empty store is equivalent to (iv) A transition function 8 that maps triples (q, a, Z) acceptance by final state. The proof is essentially the consisting of a state q, an input symbol a, and a same as it is for PDAs and so we will not repeat it here. pushdown store symbol Z, onto a finite set of instruc- The formal statement of the result is our first theorem. tions, where each instruction is in one of the following two forms: Theorem 1. A language L is accepted by some RPDA by final state if and only if it is accepted by some (1) An ordinary PDA move: (p, push a, A), where p is a (typically different) RPDA by empty store. state, a is a string of pushdown symbols, and A is As with ordinary PDAs, and for the same reasons, one of the two instructions + 1 and 0 standing for Theorem 1 does not hold for deterministic RPDAs. "advance the input head" and "do not advance the input head," respectively. (The word "push" 3 EXAMPLESAND COMPARISON TO OTHER CLASSES serves no function in the mathematics, but it does help make the notation more readable.) We next explain how RPDAs can be constructed to (2) A check-copy move: These instructions consist only accept each of the following sample languages. of a state p. (As explained in the next two defini- Examples of RPDA languages. tions, this is the state of the finite control after a L o={wwlwE{a,b}*} successful move. A successful move matches that L 1 = {wcwl w E {a, b} *} portion of the stack contents between the highest L 2 = {wh(w)l w E {a, b} *} marker $ and the top of the stack against an initial where h is the homomorphism h(a) = c and h(b) = segment of the remaining input and does so in the dde. right order for checking reduplication.) L 3 = {wxw I w ~ {a, b} * and x E {c, d} *} L4 = {al Wl Wl a2 w2 w2""an w n w n a n a n an_ I .-.a I I ale 2. An instantaneous description (ID) of an RPDA M is a {a, b}, wig {c, d} *} triple (q, w, 3"), where q is a state, w is the portion of L5 = {XlCX2 c ...cx n ca n a,_j ...a I I a i ~ {a, b} , x i E {a, input left to be processed, and 3' is the string of symbols b}*, x i is of the form ww if and only if ai = b} on the pushdown store. (The top symbol is at the left L 6 = {x! cx 2 c ...cx n ca I a 2 ...an I ai ~ {a, b}, x i E {a, end of 3", which is the same convention as that normally b}*, xi is of the form ww if and only if a i = b} used for ordinary PDA's.) L 0 is the simplest possible reduplication language. To 3. The next ID relation F is defined as follows: accept this language, all that an RPDA need do is to (t7, aw, Za) F (q, w, floO, provided (q, push/3, + l) insert the marker $ into the stack, copy symbols into ?:ffp, a, Z); the stack, guess when it has reached the midpoint, and (p, aw, Za) F (q, aw, /3a), provided (q, push/3, O) then perform a check-copy move. If the center of the 8(p, a, Z) ; string is marked with a punctuation symbol, then the (p, axw, Z3" ~ a) F (q, w, a), provided q E 8 (p, a, Z) RPDA can be deterministic, so L~ is a deterministic and ax = (Z3,) R, where (Z3')R denotes Z3" written RPDA language. backwards (so the a matches the symbol just above The language L2 illustrates the fact that reduplication the stack marker $. Note that Z3' cannot contain the need not be symbol-by-symbol identity. The RPDA to symbol $, because $ is not in the input alphabet). accept L 2 is similar to the one .that accepts Lo, except As usual, F* denotes the reflexive-transitive closure of that on reading an a it pushes a c into the stack instead F. Notice that the relation F is not a function. A given ID of an a, and on reading a b it pushes edd on the stack may have more than one next ID, and so these machines instead of b. are nondeterministic. L 3 illustrates the fact that reduplication may be 4. An RPDA is said to be deterministic provided that checked despite the intervention of an arbitrarily long (p, a, Z) contains at most one element for each triple string. The construction of the RPDA is easy. (p, a, Z). The language L 4 illustrates the facts that reduplica- 5. The language accepted by the RPDA M by final tion can be checked any number of times and that these state is defined and denoted as follows: L(M) = { w I (qo, checks may be embedded in a context free-like con- w, Z) F* (/9, A, 3") for some p E F, 3" E F*}, where q0 is struction. To accept L 4 an RPDA would push a~ and the start state and F is the set of accepting states. (A is then $ onto the stack and proceed to check for a used to denote the empty string.) If a language is reduplication w~ w~ as described for L o. If such a w~ w~ accepted by some RPDA by final state it is called an is found, that will leave just a~ on the stack. The RPDA RPDA language. If a language is accepted by some next pushes $ a 2 on the stack (the $ is on top of the a2)
Computational Linguistics, Volume 15, Number 4, December 1989 253 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication and checks for w 2 w 2. After processing an initial input stack, and when it reaches the list of verbs it would string of the form perform a check-copy move. Hence RPDAs seem ca- a 1 w t w I a2 w2 w2 ""an Wn Wn pable of weakly generating languages that exhibit the the stack will contain a~ an_ I ...at with a,, on top of the properties that keep many natural languages from being stack. It can then easily che~k for the occurrence of a context-free. As the next result indicates, their power is matching ending a~ an_ l ...al. (It is also easy to check strictly between the context-free grammars and the for an ending in the reduplicating order al a2 ""an using indexed grammars. Most of the theorem is easy to the techniques discussed below for L6.) prove, given known results. However, a proof that L 5 and L 6 illustrate the fact that reduplication can be there is an indexed language that is not an RPDA used as a distinguishable feature to carry some syntactic language will have to wait until later in this paper when or semantic information. For example, the reduplicated we will prove that the indexed language {anbnc n I n >- 0 } strings might be nouns and the reduplication might be is not an RPDA language. used to indicate a plural. The string of ais might be Theorem 2. Context-Free Languages C RPDA Lan- agreeing adjectives or verbs or whatever. L 5 using the guages C Indexed Languages (and the inclusions are mirror image construction is not meant to be typical of proper) natural language but merely to illustrate that the redu- Partial proof. The first inclusion follows from the plication might be embedded in some sort of phrase definitions. To see that the inclusion is proper recall that structure. L 6 shows that the RPDA model can obtain the { ww I w E {a, b} *}, which is well known to not be same language with cross-serial dependency instead of context-free, is an RPDA language. The second inclu- mirror imaging. sion follows from the fact that an RPDA can be simu- To accept L 6 the RPDA needs to have two marker lated by a one-way stack automaton, and all languages symbols $ in the stack at one time. To accept L 6 an accepted by one-way stack automata are indexed lan- RPDA would push the marker ~ on the stack. This first guages, as shown in Aho (1969). The proof that there is marker will eventually produce the stack contents an indexed language that is not an RPDA language will (1) a~ a n -i ""al ~ (the top is on the left) be proven in a later section of this paper.V1 which it then compares to the ending string a~ a 2 ...a~. To construct this string in the stack, it guesses the a~ 4 VARIATIONSON THE RPDA MODEL and uses a second marker to check its guesses. For example, if the RPDA guesses that a~ = b then it pushes There are a number of variations on the RPDA model a~ = b onto the stack and proceeds to check that Xl is a that one might try. One tempting variant is to replace reduplication string. To do this it pushes another marker the check-copy move with a stack-flipping move. This onto the stack and checks x I in the way described for variant would allow the entire stack contents above the L o and other languages. If x~ does not check out, then marker ~ to be flipped so that, in one move, the the computation aborts. If x t does check out, the stack pushdown-store contents a ~ T would change to c~ $ T. contains a 1 ~, (the top is on the left) and the RPDA then (The "top" is always the left end.) This would allow the guesses a 2. Say it guesses that a 2 ----- a and hence must machine to check for reduplication. However, it also check that a 2 is not of the form ww. The RPDA then allows the machine to check for everything else. This pushes a 2 onto the stack and performs the check. One flipping stack variant is equivalent to a Turing machine straightforward way to perform the check is to push a because it can simulate a Turing machine tape by marker ~ on the stack and then read the first half of x2 continuaUy flipping the stack and using its finite control guessing at where it differs from the second half. The to "rotate" to any desired stack position. For example, RPDA pushes its guess of the second half on the stack. af'~ ~, 'y can be transformed into fla $ T by moving one If the RPDA correctly guesses the second half and if it symbol at a time from the top of the stack to just above ensures that it guesses at least one difference from the the n~tarker $. The top symbol is moved by remember- first half, then x2 checks out. If x2 checks out, then the ing the symbol in the finite-control, flipping the stack, stack will contain a 2 a l $ after all this. Proceeding in placing the remembered symbol on the stack, and this way, the RPDA obtains the stack contents shown in flipping again. (1) and then performs a final check-copy move to see if One way to avoid the "Turing tar pit" in this flipping it matches the rest of the input string. stack variant would be to deprive the machine of its By examining these examples it is easy to see how an stack marker $ after a flip. This would appear to limit RPDA could deal with the reduplication constructions the number of flips and so prevent the Turing machine from natural language that were mentioned in the intro- simulation outlined. However, a nondeterministic ma- duction. For example, in the Swiss-German subordinate chine; could simply place a large supply of markers in clause construction, an RPDA would first push the the stack so that when one was taken away another stack marker ~ onto the stack, then it would read the would be at hand. To foil this trick, one might limit that list of nouns, then for each noun it would nondetermin- machine to one stack marker, but this would restrict the istically choose a verb that requires the case of that machine so that it cannot naturally handle reduplica- noun. It would then push the chosen verb onto the tions nested inside of reduplications. In Section 8,
254 Computational Linguistics, Volume 15, Number 4, December 1989 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication
RPDAs with only a single marker (and possessing one guages. (A Full AFL is any class of languages that other restriction at the same time) are studied. That contains at least one nonempty language and that is section concludes with a discussion of why some natu- closed under union, A-free concatenation of two lan- ral language constructs appear to require multiple mark- guages, homomorphism, inverse homomorphism, and ers. intersection with any finite-state language. See Salomaa When using a copy-check move, an RPDA can read 1973, for more details.) an arbitrarily long piece of input in one move. This The notion of a finite-state transduction is important definition was made for mathematical convenience. A when analyzing pushdown machines. If a finite-state realistic model would have to read input one symbol per control reads a string of input while pushing some string unit time. However, the formal model does not seri- onto the stack (without any popping), then the string in ously misrepresent realistic run times. If the model were the stack is a finite-state transduction of the input string. changed to read the input one symbols at a time while Unfortunately, the concept of a finite-state transduction emptying the stack contents above the marker, then the is fading out of the popular textbooks. We will therefore run time would at most double. give a brief informal definition of the concept. Definition. A finite-state transducer is a nondetermin- 5 CLOSUREPROPERTIES istic finite-state machine with output. That is, it is a finite-state machine that reads its input in a single The next two theorems illustrate the fact that RPDA left-to-right pass. In one move it does all of the follow- languages behave very much like context-free lan- ing: either read a symbol or move without consuming guages. The proofs are straightforward generalizations any input (called moving on the empty input) and then, of well-known proofs of the same results for context- on the basis of this symbol or the empty input, as well free languages. Using the PDA characterization of con- as the state of the finite-state machine, it changes state text-free languages and the proofs in that framework, and outputs a string of symbols. (If, on first reading, you one need do little more than replace the term "PDA" by ignore moving on the empty string, the definition is very "RPDA" to obtain the corresponding results for RPDA easy to understand. Moving on the empty string simply languages. allows the transducer to produce output without reading Theorem 3. The class of RPDA languages is closed any input.) There is a designated start state and a set of under the following operations: intersection with a designated accepting states. finite-state language, union, star closer, and finite-state In a computation of a finite-state transducer there is transduction (including the special cases of homomor- an output string consisting of the concatenation of the phism and inverse homomorphism). individual strings output, but not all output strings are considered valid. The computation begins in the start Theorem 4. The class of deterministic RPDA lan- state and is considered valid if and only if the compu- guages is closed under the operations of intersection tation ends in one of a designated set of accepting with a finite-state language and complement. states. A string y is said to be a finite-state transduction One detail of the proof of Theorem 3 does merit some of the string x via the finite-state transducer T provided mention. It may not, at first glance, seem obvious that that there is a valid computation of T with input x and the class of RPDA languages is closed under intersec- output y. tion with a regular set, since the proof that is usually To motivate the following definition, consider chang- used for ordinary PDAs does not carry over without ing a language so that plurals are marked by reduplica- change. In the ordinary PDA case, all that is needed is tion. For example, in English "papers" would change to have a finite-state machine compute in parallel with to "paperpaper." A sentence such as "Writers need the PDA. In the case of an RPDA this is a bit more pens to survive" would change to "Writerwriter need complicated, since the RPDA does not read its input penpen to survive." This change of English can be symbol by symbol. In a check-copy move, an RPDA obtained by first replacing all plural nouns with a special can read an arbitrarily long string in one move. How- symbol (a in the definition) and then performing a ever, the finite-state control can easily keep a table reduplication substitution as described in the definition. showing the state transitions produced by the entire If the change were other than an exact copy, it would string above the marker $. When a second ~ is inserted still satisfy the definition provided that a finite-state into the stack, the old transition table is stored on the machine could compute the approximate copy. This stack and a new transition table is started. The other operation allows us to take a language and introduce details of the proof are standard. reduplication in place of any suitably easily recognized Theorem 3 implies that the class of RPDA languages class of words and so obtain a language that uses is a full AFL (Abstract Family of Languages), which in reduplication to mark that class. The following theorem some circles invests the class with a certain respectabil- says that RPDA languages are closed under these sorts ity. This is because such closure properties determine of operations. much of the character of well-known language classes, Definition. Let L be a language, a a symbol, and T a such as context-free languages and finite-state lan- finite-state transduction. Define the language L' to
Computational Linguistics, Volume 15, Number 4, December 1989 255 Walter J. Savitch A Formal ldodd for Context-Free Languages Augmented with Reduplication consist of all strings w that can be written in the form and then define s 1 and s 3 by the equation s = sl s,2 s3. Xo Yo xl Yl ""Xn -1 Yn -I Xn where Define r z to be the portion of input consumed while (i) each xi contains no a, s pushing s2 onto the stack, and then define r~ and r 3 by (ii) Xo axl a ""Xn -I ax, E L, and the equation r = r 1 r z r 3. It is then straightforward to (iii) each Yi is of the form vv' where v' is a finite-state show that the conclusion of the lemma holds.[] transduction of v via T. Our second pumping lemma for RPDAs draws the A language L', obtained in this way, is called a redupli- same conclusion as a weak form of the pumping lemma cation substitution of the language L via T by substituting for context-free languages. Since the pumping lemma reduplication strings for a. More simply, L' is called a for context-free languages will be used in the proof of reduplication substitution of L provided there is some the second pumping lemma for RPDAs, we reproduce symbol a and some such finite-state transduction T such the context-free version for reference. that L' can be obtained from L in this way. Weak Pumping Lemma for CFLs. If L is a context- Theorem 5. If L' is a reduplication substitution of a free language, then there is a constant k, depending on context-free language, then L' is an RPDA language. L, :such that the following holds: If z E L and length (z) Among other things, Theorem 5 says that if you add > k, then z can be written in the form z = uvwxy where reduplication to a context-free language in a very simple either v or x is nonempty and uv g wx i y E L for all i -> 0. way, then you always obtain an RPDA language. In The following version of the pumping lemma for Section 8, we will prove a result that is stronger than RPDAs makes the identical conclusion as the above Theorem 5 and so we will not present a proof here. pumping lemma for context-free languages. ]Pumping Lemma 2. If L is an RPDA language, then 6 NoNRPDA LANGUAGES there is a constant k, depending on L, such that the To prove that certain languages cannot be accepted by following holds: If z E L and length (z) > k, then z can any RPDA, we will need a technical lemma. Despite the be written in the form z = uvwxy, where either v or x is messy notation, it is very intuitive and fairly straight- nonempty and lgV i WX i y E L for all i -> 0. forward to prove. It says that if the stack marker ~ is Proof. Let M be an RPDA accepting L and let k be as used to check arbitrarily long reduplications, then these in the Pumping Lemma 1. We decompose L into two reduplications can be pumped up (and down). For languages so that L = L 1 U L 2. Define L 1 to be the set example, suppose an RPDA accepts a string of the form of all strings z in L such that the Pumping Lemma 1 usst and does so by pushing $, then s onto the stack, applies to at least one accepting computation on z. In and then matching the second s by a check-copy move. other words, z is in LI if and only if there is a It must then be true that, if s is long enough, then s can computation of M of the form be written in the form s I s 2 s 3 and for all i > 0, the RPDA (qo', Z, Zo) F* (Pl, rst, $ fl) F* (P2, st, s n $ [3) F can do a similar pushing and checking of sls2gs3 to (P3., t, /3) F* (pf, A, T) accept USIS2 i S 3 S 1 S2 i s3t. where qo is the start state, pf is an accepting state, and Pumping Lemma 1. For every RPDA M, there is a length(s) > k. By Pumping Lemma 1, it follows that the constant k such that the following holds. If length(s) > conclusion of Pumping Lemma 2 applies to all strings in k and L 1. (In this case we can even conclude that x-is always (Pl, rst, ~ [3) F* (P2, st, s n $ fl ) F (P3, t, ~), nonempty. However, we will not be able to make such where the indicated string $/3 is never disturbed, then a conclusion for strings in L2.) r and s may be decomposed into r = r~ rE r3 and s = L 2 is defined as the set of all strings accepted by a s~ s2 s3, such that s2 is nonempty and for all i - O, (pl, particular ordinary PDA M 2 that simulates many of the r I rEi r 3 s I Sz i S 3 t, $ /3) F* (P2, S1 Szi S3 t, (S I $2 i S3) R ~¢ /3) computations of the RPDA M. Define M2 to mimic the I- (P3, t, /3). computation of M but to buffer the top k + 1 symbols of Proof. Without loss of generality, we will assume that the stack in its finite-state control, and make the follow- M pushes at most one symbol onto the stack during ing modifications to ensure that it is an ordinary PDA: if each move. Let k be the product of the number of states M2 has to mimic a check-copy move that matches k or in M and the number of stack symbols of M. Consider fewer symbols above the marker $, it does so using the the subcomputation stack buffer in its finite-state control. If M2 ever needs (191, rst, $/3) F* (P2, st, s R ~ /3) to mimic a check-copy move that matches more than k Let s = a~a 2 ... a m where the a i are single symbols and stack symbols above the $, it aborts the computation in m > k. Let q~, q2 ..... qm be the state of M after it places a nonaccepting state. M 2 can tell if it needs to abort a this occurrence of ai onto the stack so that, after this computation by checking the finite stack buffer in its point, ag is never removed from the stack until after this finite-state control. If the buffer contains k + 1 symbols subcomputation. Since m > k, there must be i < j such but no marker $, and if M would do a check-copy that a i = aj and qg = qj . Set move, then ME aborts its computation. $2 : ai + 1 ai + 2 ... aj By definition, L = L~ U L 2 . (The sets L~ and L 2 need
256 Computational Linguistics, Volume 15, Number 4, December 1989 Waiter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication
not be disjoint, since M may be nondeterministic, and To see the limits of this metatheorem, note that the hence, a given string may have two different accepting language { ww I w ~ {a, b} *} is an RPDA language, and computations. However, this does not affect the argu- so to prove that it is not a context-free language, we ment.) Because it is accepted by an ordinary PDA, L 2 is should need more than the Weak Pumping Lemma. a context-free language, and the Pumping Lemma for Indeed, one cannot get a contradiction by assuming context-free languages holds for it and some different k. only that the Weak Pumping Lemma applies to this Hence, by redefining k to be the maximum of the ks for language. A proof that this language is not context-free LI and L2, we can conclude that the Pumping Lemma 2 must use some additional fact about context-free lan- holds for L = L~ U L2 and this redefined k.[~ guages, such as the fact that we can assume that The next theorem and its proof using the pumping length(vwx) <_ k, where vwx is as described in the lemma illustrates the fact that RPDAs, like context-free pumping lemma. grammars, can, in some sense, check only "two things The metatheorem is another indication that the at a time." RPDA languages are only a small extension of the context-free languages. If it is easy to prove that a Theorem 6. L = {a nb ncnln_>0}isnotanRPDA language is not context-free (i.e., if the language is language. "very noncontext-free"), then the language is not an Proof. Suppose L is an RPDA language. We will RPDA language either. derive a contradiction. By Pumping Lemma 2, there is a value of n and strings u, v, w, x, and y such that a n b n c n 7 A CONSTRUCTION MISSED BY THE MODEL = uvwxy with either v or x nonempty and such that uv i wx i y E L for all i > 0. A straightforward analysis of the As we have already noted, both Dutch and Swiss- possible cases leads to the conclusion that uv 2 wx 2 y is German contain constructions consisting of a string of not in L, which is the desired contradiction.if] nouns followed by an equal (or approximately equal) Since it is known that the language L in Theorem 6 is number of verbs. Hence these languages contain sub- a TAG language, and since the TAG languages are strings of the form included in the indexed languages, we obtain the follow- N l N 2 ...N n V, V 2 ...V,, ing corollaries. The second corollary is the promised In the case of Swiss-German, additional agreement completion of the proof of Theorem 2. rules suffice to show that these constructions are be- Corollary. There is a TAG language that is not an yond the reach of context-free grammar, although not RPDA language. beyond the reach of RPDAs. (See the discussion of Shieber 1985 earlier in this paper.) Because Dutch lacks Corollary. There is an indexed language that is not an the strong agreement rule present in Swiss German, the RPDA language. same proof does not apply to Dutch. Manaster-Ramer There are many versions of the pumping lemma for (1987) describes an extension of this construction within context-free languages. (See Ogden 1968; Harrison Dutch and argues that this extension takes Dutch be- 1978; Hopcroft and Ullman 1979.) Most versions make yond the weak generative capacity of context-free the additional conclusion that length(vwx) <- k. Often grammar. Although we are admittedly oversimplifying one can prove that a language is not context-free the data, the heart of his formal argument is that two without using this additional conclusion about the such strings of verbs may be conjoined. Hence, Dutch length of vwx. In other words, we can often prove that contains substrings that approximate the form a language is not context-free by using only the weak Nl N2 ...Nn VI V2 ...Vn en ('and') V i V2 ""Vo form of the pumping lemma given above. One such The Dutch data support only the slightly weaker claim language is the one given in Theorem 6. If you review that the number of nouns is less than or equal to the the proof of Theorem 6, then you will see that all we number of verbs. Hence, Manaster-Ramer's argument needed was the Pumping Lemma 2. Moreover, that is, in essence, that Dutch contains a construction simi- pumping lemma has the identical conclusion as that of lar to the following formal language: the Weak Pumping Lemma for context-free languages. g = {ail~cJli~j} This leads us to the following informal metatheorem: He uses this observation to argue, via the Pumping Lemma for Context-Free Languages, that Dutch is not Metatheorem. If L can be proven to not be context- a context-free language. A careful reading of his argu- free via the Weak Pumping Lemma for CFLs, then L is ment reveals that, with minor alterations, the argument not an RPDA language. can be made to work using only the Weak Pumping This is not an official theorem since the phrase "via Lemma. Hence by the metatheorem presented here (or the Weak Pumping Lemma" is not mathematically a careful review of his proof), it follows that his argu- precise. However, the metatheorem is quite clear and ment generalizes to show that the language L is not an quite clearly valid in an informal sense. It can be made RPDA language. Hence, if one accepts his data, the precise, but that excursion into formal logic is beyond same argument shows that Dutch is not an RPDA the scope of this paper. language.
Computational Linguistics, Volume 15, Number 4, December 1989 257 Walter J. Savitch A Formal Mode~ for Context-Free Languages Augmented with Reduplication
The RPDA model could be extended to take account text-free grammars. If we take T to be the transduction of this and similar natural language constructions that accepts only the empty string as input and output, missed by the model. One possibility is simply to allow then the set of productions associated with the schema the RPDA to check an arbitrary number of input strings (A -~ ct, I) consists of the single context-free production to see if they are finite-state transductions of the string A -~ a. In particular, every context-free grammar is above the marker $. There are a number of ways to do (except for notational detail) an RCFG. this. However, it seems preferable to keep the model Recall that a context-free grammar in Greibach Nor- clean until we have a clearer idea of what constructions, mal Form is one in which each production is of the form other than reduplication, place natural language beyond A --~ act the reach of context-free grammar. The RPDA model, where a is a terminal symbol and ct is a string consisting as it stands, captures the notion of context-free gram- entirely of nonterminals. It is well known that every mar plus reduplication, and that constitutes one good context-free language can be (weakly) generated by a approximation to natural language string sets. context-free grammar in Greibach Normal Form. The schemata described in the definition of RCFGs have 8 REDUPLICATION GRAMMARS some similarity to context-free rules in Greibach Nor- mal Form, except that they start with a reduplication Although we do not have a grammar characterization of string, rather than a single terminal symbol, and the RPDA languages, we do have a grammar class that is an remaining string may contain terminal symbols. Also extension of context-free grammar and that is adequate the leading reduplication string may turn out to be the for a large subclass of the RPDA languages. The model empty string. Thus, these are very far from being in consists of a context-free grammar, with the addition Greibach Normal Form. Yet, as the proof of the next that the right-hand side of rewrite rules may contain a result shows, the analogy to Greibach Normal Form can location for an unboundedly long reduplication string of sometimes be productive. terminal symbols (as well as the usual terminal and nonterminal symbols). Theorem 7. If L is a reduplication substitution of a Definition. A reduplication context-free grammar context-free language, then there is an RCFG G such (RCFG) is a grammar consisting of terminal, nontermi- that L = L(G). nal, and start symbols as in an ordinary context-free Proof. Let G' be a context-free grammar, T a finite- grammar, but instead of a finite set of productions, it state transduction and a a symbol such that L is has a finite set of rule schemata of the following form: obtained from L(G') via T by substituting reduplication (A --~ ct, T) where A is a nonterminal symbol, a is a strings for a. Without loss of generality, we can assume string of terminal and/or nonterminal symbols, and that G' is in Greibach Normal Form. The RCFG G where T is a finite-state transducer. (Thus, A ~ a is an promised in the theorem will be obtained by modifying ordinary context-free rule, but it will be interpreted G'. To obtain G from G' we replace each G' rule of the differently than normal.) form The production set associated with the schema (A --~ A --~ aA 1 A 2 ...A n, a, T) is the set of all context-free rules of the form: A --* where a is the symbol used for the reduplication substi- ww'ct, where w is a string of terminal symbols, and w' is tution, by the schema obtained from w by applying the finite-state transduc- (A --~ Al A2 ""An, T) tion T to the string w. The remaining rules of G' are left unchanged except for The next step relation ff for derivations is defined as the technicality of adding a finite-state transduction that follows: accepts only the empty string as input and output, and a ~ /3 if there is some context-free rule in some so leaves the rule unchanged for purposes of generation. production set of some rule schema of the grammar such that a ~ /3 via this rule in the usual manner for A routine induction then shows that the resulting RCFG context-free rewrite rules. As usual, ~ is the reflexive- G is such that L(G) = L.[3 transitive closure of ~. Parsing with an RCFG does not require the full The language generated by an RCFG, G, is defined power of an RPDA, but only requires the restricted type and denoted in the usual way: L(G) = { w Lw a string of of R]PDA that is described next. terminal symbols and S ~ w}, where S is the start Definition. A simple RPDA is an RPDA such that, in symbol. any computation: Notice that an RCFG is a special form of infinite (i) there is at most one occurrence of the marker $ in context-free grammar. It consists of a context-free the stack at any one time, and grammar with a possibly infinite set of rewrite rules, (ii) as long as the marker symbol $ is in the stack, the namely the union of the finitely many production sets RPDA never removes a symbol from the stack. associated with the schemata. However, there are very More formally, an RPDA M is a simple RPDA severe restrictions on which infinite sets of productions provided that the following condition holds: if the are allowed. Also notice that RCFGs generalize con- instruction (p, push a, A) E 8(q, a, Z) can ever be used
258 Computational Linguistics, Volume 15, Number 4, December 1989 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication
when $ is in the stack, then a = /3 Z for some/3 and a successful check-copy move. M' mimics M step by does not occur in a. step as long as M would not have the marker $ in the Like Theorem 1, the following equivalence for simple stack and as long as the input is not one of the new RPDAs is trivial to prove by adapting the proof of the symbols
Computational Linguistics, Volume 15, Number 4, December 1989 259 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication have complicated phrase structures embedded within a REFERENCES reduplication construction indicates that simple RPDAs may not be adequate for natural language syntax. If one Aho, Alfred V. 1968 Indexed Grammars--an Extension to Context Free Grammars. Journal of the Association for Computing Ma- assumes a language like English but with syntactic chinery 15: 647-671. gender, strong agreement rules, and a well-behaved use Aho, Alfred V. 1969 Nested-Stack Automata. Journal of the Associ- of respectively, then one can easily see why one might ation for Computing Machinery 16: 383-406. want more power than that provided by a simple RPDA. Bar-HiUel, Y. and Shamir, E. 1964 Finite State Languages: Formal An example of a kind of sentence that seems beyond the Representations and Adequacy Problems. In: Bar-Hillel, Y. (ed.), reach of simple RPDAs is the following: Language and Information. Addison-Wesley, Reading, MA: 87-98. Bre, snan, J.; Kaplan, R. M.; Peters, S. and Zaenen, A. 1982 Cross- Tom, who has had three wives, Sally, who has had Serial Dependencies in Dutch. Linguistic Inquiry 13: 613-635. seven husbands, Mary, who lost John, Hank, and Culy, Christopher 1985 The Complexity of the Vocabulary of Bam- Sammy to cancer, heart disease, and stroke, respec- bara. Linguistics and Philosophy 8: 345-351. tively, and Frank, who had only one wife and lost her Gazdar, Gerald 1985 Applicability of Indexed Grammars to Natural last January, are a widower, widow, widow, and Languages, Report No. CSLI-85-34. Center for the Study of Language and Information. Palo Alto, CA. widower, respectively. Gazdar, Gerald and Pullum, Geoffrey K. 1985 Computationally Rel- The natural way to handle these sorts of sentences with evant Properties of Natural Languages and their Grammars. New an RPDA is to have two markers + in the stack at once, Generation Computing 3: 273-306. and we conjecture that a single marker will not suffice. Harrison, Michael A. 1978 Introduction to Formal Language Theory. English does not have the syntactic gender and Addison-Wesley, Reading, MA. strong agreement rules that would allow us to prove, via Hopcroft, John E. and Ullman, Jeffrey D. 1979 Introduction to Automata Theory, Languages, and Computation. Addison-Wes- this construction, that English is not context-free. We ley. Reading, MA. merely put it forth as an example of a potential natural Huybregts, M. A. C. 1976 Overlapping Dependencies in Dutch. language situation. Utrecht Working Papers in Linguistics l: 24-65. Joshi, Aravind K. 1985 Tree Adjoining Grammars: How Much Context-Sensitivity is Required to Provide Reasonable Structural 9 SUMMARY Descriptions? In: Dowty, D. R.; Karttunen, L. and Zwicky, A. M. (eds.), Natural Language Processing: Psycholinguistic, Compu- We have seen that the RPDA model is very similar to tational, and Theoretic Perspectives. Cambridge University Press, the PDA characterization of context-free languages. New York, NY. Thus from an automata theoretic point of view, RPDA Joshi, Aravind K. 1987 An Introduction to Tree Adjoining Grammars. In: Manaster-Ramer, A. (ed.), Mathematics of Language. John languages are very much like context-free languages. Benjamins, Philadelphia, PA: 87. We have seen that both classes have similar closure Joshi, Aravind K. 1989 Processing Crossed and Nested Dependen- properties, and so they are similar from an algebraic cies: An Automaton Perspective on the Psycholinguistic Results. point of view as well. Moreover, the context-free lan- Preprint, Department of Computer and Information Sciences, guages and the RPDA languages have similar pumping University of Pennsylvania, Philadelphia, PA. lemmas that exclude many of the same unnatural lan- Joshi, Aravind K.; Vijay-Shanker, K. and Weir, D. J. 1989 The Convergence of Mildly Context-Sensitive Grammar Formalisms. guage sets and even exclude them for the same reasons. Report MS-CIS-89-14, LINC LAB 144. Department of Computer Hence, the class of RPDAs are only mildly stronger anci Information Science, University of Pennsylvania, Philadel- than context-free grammars. However, the model is phia, PA. sufficiently strong to handle the many reduplication Kac, lvl. B., Manaster-Ramer, A. and Rounds, W. C. 1987 Simulta- constructions that are found in natural language and that neous-Distributed Coordination and Context-Freeness. Computa- seem to place natural language outside of the class of tional Linguistics 13: 25-30. Manaster-Ramer, Alexis 1983 The Soft Formal Underbelly of Theo- context-free languages. The RPDA languages do not, as retical Syntax. Papers from the Nineteenth Regional Meeting. yet, have a grammar characterization similar to that of Chicago Linguistic Society 254-262. context-free grammar, but the RCFG grammars are Manaster-Ramer, Alexis 1986 Copying in Natural Languages, Con- context-free like grammars that do capture at least a text-Freeness, and Queue Grammars. Proceedings of the 24th large subclass of the RPDA languages. Annual Meeting of the Association for Computational Linguistics 85-89. Manaster-Ramer, Alexis 1987 Dutch as a Formal Language. Linguis- ACKNOWLEDGMENTS tics and Philosophy 10: 221-246. Ogden, W. F. 1968 A Helpful Result for Proving Inherent Ambiguity. This research was supported in part by NSF grant DCR-8604031. Bill Mathematical Systems Theory 2: 191-194. Chen and Alexis Manaster-Ramer provided a number of useful Pollard, Carl J. 1984 Generalized Phrase Structure Grammars, Head discussions on this material. A preliminary version of this work was Grammars, and Natural Languages. Ph.D. thesis, Stanford Uni- presented at the Workshop on Mathematical Theories of Language, versity, Stanford, CA. LSA Summer Institute, Stanford University, Summer 1987. Com- Postail, Paul 1964 Limitations of Phrase Structure Grammars. In: ments of the workshop participants, particularly those of Aravind Fodor, J.A.; and Katz, J.J. (eds.), The Structure of Language: Joshi, K. Vijay-Shanker, and David Weir, helped shape the current Readings in the Philosophy of Language. Prentice-Hall, Engle- version of this work. A number of remarks by two anonymous wood Cliffs, N.J.: 137-151. referees also help in the preparation of the final draft of this paper. I Pullum, Geoffrey K. and Gazdar, G. 1982 Natural Languages and express my thanks to all these individuals for their help in this work. Context Free Languages. Linguistics and Philosophy 4: 471-504.
260 Computational Linguistics, Volume 15, Number 4, December 1989 Walter J. Savitch A Formal Model for Context-Free Languages Augmented with Reduplication
Roach, Kelly 1987 Formal Properties of Head Grammars. In: Man- Steedman, M. J. 1988 Combinators and Grammars. In: Oehrle, R.; asteroRamer, A. (ed.), Mathematics of Language. John Ben- B.ach, E.; and Wheeler, D. (eds.), Categorial Grammars and jamins, Philadelphia, PA : 293-347. Natural Language Structures. Reidel, Dordrecht, The Nether- Rounds, William C.; Manaster-Ramer, A.; and Friedman, J. 1987 lands : 417--442. Finding Natural Language a Home in Formal Language Theory. In: Manaster-Ramer, A. (ed.), Mathematics of Language. John Vijay-Shanker, K.; Weir, D. J.; and Joshi, A. K. 1987 On the Benjamins, Philadelphia, PA : 87-114. Progression from Context-Free to Tree Adjoining Languages. In: Salomaa, A. 1973 Formal Languages. Academic Press, New York, Manaster-Ramer, A. (ed.), Mathematics of Language, John Ben- NY. jamins, Philadelphia, PA : 389-401. Shieber, Stuart M. 1985 Evidence Against the Context-Freeness of Weir, D.J.; Vijay-Shanker, K.; and Joshi,A. K. 1986 The Relationship Natural Language. Linguistics and Philosophy 8: 333-343. of Tree Adjoining Grammars and Head Grammars. Proceedings of Steedman, M. J. 1987 Combinatory Grammars and Parasitic Gaps. the 24th Annual Meeting of the Association for Computational Natural Language and Linguistic Theotw 5: 403-439. Linguistics, New York, NY : 67-74.
Computational Linguistics, Volume 15, Number 4, December 1989 261