Position-and-Length-Dependent Context-Free Grammars – A New Type of Restricted Rewriting fCFL flCFL fldCFL eREG fedREG REG feREG CFL lCFL feCFL fedCFL fledCFL fleCFL fREG eCFL leCFL ledCFL Vom Fachbereich Informatik der Technischen Universität Kaiserslautern zur Erlangung des akademischen Grades Doktor der Naturwissenschaften (Dr. rer. nat) genehmigte Dissertation von Frank Weinberg Dekan: Prof. Dr. Klaus Schneider Vorsitzender der Prüfungskommission: Prof. Dr. Hans Hagen 1. Berichterstatter: Prof. Dr. Markus Nebel 2. Berichterstatter: Prof. Dr. Hening Fernau Datum der wissenschaftlichen Aussprache: 3. März 2014 D 386 Abstract For many decades, the search for language classes that extend the context-free laguages enough to include various languages that arise in practice, while still keeping as many of the useful properties that context-free grammars have – most notably cubic parsing time – has been one of the major areas of research in formal language theory. In this thesis we add a new family of classes to this field, namely position-and-length- dependent context-free grammars. Our classes use the approach of regulated rewriting, where derivations in a context-free base grammar are allowed or forbidden based on, e.g., the sequence of rules used in a derivation or the sentential forms, each rule is applied to. For our new classes we look at the yield of each rule application, i.e. the subword of the final word that eventually is derived from the symbols introduced by the rule application. The position and length of the yield in the final word define the position and length of the rule application and each rule is associated a set of positions and lengths where it is allowed to be applied. We show that – unless the sets of allowed positions and lengths are erally complex – the languages in our classes can be parsed in the same time as context-free grammars, using slight adaptations of well-known parsing algorithms. We also show that they form a proper hierarchy above the context-free languages and examine their relation to language classes defined by other types of regulated rewriting. We complete the treatment of the language classes by introducing pushdown automata with position counter, an extension of traditional pushdown automata that recognizes the languages generated by position-and-length- dependent context-free grammars, and we examine various closure and decidability properties of our classes. Additionally, we gather the corresponding results for the subclasses that use right-linear resp. left-linear base grammars and the corresponding class of automata, finite automata with position counter. Finally, as an application of our idea, we introduce length-dependent stochastic context-free grammars and show how they can be employed to improve the quality of predictions for RNA secondary structures. iii iv Acknowledgements Of course, producing such a thesis is usually not possible without the support of various people. In the case of this thesis, the major supporters are the Algorithms and Complexity Group of Prof. Markus Nebel. They did not only provide a very pleasant environment to work in – so pleasant that it actually proved difficult to cut myself loose from it – but they also allowed many helpful discussions, not only on the topics of the thesis, helping to find or evaluate new ideas, but also on other topics, helping to free the mind from failed approaches. A special “Thank You!” goes to Raphael Reitzig and Anne Berres, both of whom made numerous helpful suggestions when proofreading – or in some places rather fighting through – earlier drafts of this document. v vi Contents 1. Introduction 1 1.1. Overview............................................1 1.2. Notation.............................................2 1.3. Motivation............................................2 1.4. Known Forms of Restricted Rewriting.............................3 1.4.1. Conditions on Sentential Forms............................3 1.4.2. Restrictions on the Rule Sequence...........................4 1.4.3. Coupled Nonterminal Symbols.............................5 1.4.4. Overview........................................6 1.5. New Idea: Position and Length Restrictions..........................6 2. Definitions and Basic Results9 2.1. Grammars............................................9 3. Automata 19 3.1. Finite Automata......................................... 19 3.2. Pushdown Automata...................................... 23 4. Parsing 29 4.1. Parsing the Regular-Based Language Classes......................... 29 4.2. Parsing the context-free-based Language Classes....................... 30 4.2.1. CYK Algorithm.................................... 30 4.2.2. Valiant’s Algorithm................................... 31 4.2.3. Earley’s Algorithm................................... 33 5. Hierarchy of Language Classes 37 5.1. Internal Hierarchy........................................ 39 5.2. Relation to Other Classes.................................... 52 6. Decidability Properties 57 7. Closure Properties 59 8. Application: Prediction of RNA Secondary Structures 65 8.1. Problem Setting......................................... 65 8.2. Formal Definitions....................................... 66 8.3. Estimating Rule Probabilities.................................. 69 8.4. Determining the Most Probable Derivation........................... 74 8.4.1. CYK Algorithm.................................... 74 8.4.2. Valiant’s Algorithm................................... 75 8.4.3. Earley’s Algorithm................................... 75 8.5. Experiments........................................... 76 8.5.1. Data........................................... 77 8.5.2. Grammars........................................ 77 8.5.3. Observations and Dicussion.............................. 78 vii Contents 8.5.4. Runtime......................................... 79 8.5.5. Second Experiment................................... 79 9. Conclusion and Outlook 81 9.1. Possible Future Work...................................... 81 9.1.1. Other base grammars.................................. 81 9.1.2. Extending the Application............................... 82 Bibliography 83 A. Index of Notations 89 Lebenslauf des Verfassers 95 viii 1. Introduction In this thesis, we will introduce position-and-length-dependent context-free grammars, a new type of formal grammars based on the abstract idea of restricted rewriting. The basic idea of restricted rewriting is to extend the generative power of a simple grammar class – typically the context-free grammars – by restricting the applicability of the rules based on conditions that may refer to things as the current sentential form or the sequence of rules applied before. As an example, consider context-sensitive grammars. In these, each rule replaces a single nonterminal symbol, just as the rules of a context-free grammar do. But in the context-sensitive grammar the rule may only be applied if the replaced nonterminal is surrounded by a specific context that is given with the rule. In this example the condition on a rule is local, i.e., we can decide if a rule is applicable to a specific nonterminal in a sentential form by looking at the sentential form alone. Other types of restricted rewriting have global constraints. For example, in unordered vector grammars, the rules are grouped into sets, called vectors, and a derivation is valid, if all the rules in a vector have been used equally often in the derivation. (The number of applications may differ between vectors.) For position-and-length-dependent context-free grammars we use a mixed approach. While the validity is checked for each individual rule application, the conditions depend on the derivation as a whole and thus can only be verified for a complete derivation. Specifically, we look at the yield of a rule application, i.e. the subword that is eventually derived from the symbols introduced by the rule. We then allow or forbid the rule application based on position and length of this yield, where the position is defined by the number of characters in front of resp. behind the yield. 1.1. Overview The remaining sections of this chapter introduce – after a short explanation of some notational conventions we use – the motivation that prompted the research of restricted rewriting along with several of the approaches that have been previously examined. Then, we give an informal overview of position-and-length-dependent grammars and the subclasses we are going to examine. These subclasses result from allowing fewer restrictions or using right- resp. left-linear base grammars. We formally introduce the grammar classes in Chapter2. There we also give some basic results, e.g. the fact that our grammars can be transformed into equivalent ones in Chomsky Normal Form. In Chapter3, we define equivalent automata for each of the classes. Unsurprisingly, they result from adding position checks and/or length checks to pushdown automata resp. finite automata in a suitable way. We also show that the same language classes result from right- and left-linear grammars, though the type of restrictions needed to arrive at some class can differ for the two grammar types. In Chapter4, we describe how several well-known parsing algorithms can be adapted to the new grammar classes. This can be done without using additional space or time except what is necessary to verify membership in the restriction sets for triples encountered during the parsing. This is not immediately obvious since, other than for traditional CFG, the conditions