INllHHXXTION AL'lttMATA TIH-'OKY, Kan k ImnncK Kmn D bl I viy KTUNOTES.IN Downloaded from Ktunotes.in INTRODUCTION TO AUTOMATA THEORY, LANGUAGES, COMPUTATION JOHN E. HOPCROFT Cornell University JEFFREY D. ULLMAN Princeton University KTUNOTES.IN AD DI SON-WESLEY PUBLISHING COMPANY Reading, Massachusetts Menlo Park, California London • Amsterdam • Don Mills, Ontario • Sydney Downloaded from Ktunotes.in This book is in the ADDISON -WESLEY SERIES IN COMPUTER SCIENCE Michael A. Harrison, Consulting Editor Library of Congress Cataloging in Publication Data Hopcroft, John E. , 1939- Introduction to automata theory, languages, and computation. Bibliography: p. Includes index. 1. Machine theory. 2. Formal languages. 3- Computational complexity. I. Ullman, Jeffrey D. , 19^2- joint author. II. Title. QA267.H56 629.8'312 78-67950 KTUNOTES.INISBN 0-201-02988-X Copyright (O 1979 by Addison-Wesley Publishing Company, Inc. Philippines copyright 1979 by Addison-Wesley Publishing Company, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photoc6pying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. Published simultaneously in Canada. Library of Congress Catalog Card No. 78-67950. ISBN: 0-201-02988-X LMNOPQ-DO-89876 Downloaded from Ktunotes.in PREFACE Ten years ago the authors undertook to produce a book covering the known material on formal languages, automata theory, and computational complexity. In retrospect, only a few significant results were overlooked in the 237 pages. In writing a new book on the subject, we find the field has expanded in so many new directions that a uniform com- prehensive coverage is impossible. Rather than attempt to be encyclopedic, we have been brutal in our editing of the material, selecting only topics central to the theoretical development of the field or with importance to engineering applications. OverKTUNOTES.INthe past ten years two directions of research have been of paramount im- portance. First has been the use of language-theory concepts, such as nondeterminism and the complexity hierarchies, to prove lower bounds on the inherent complexity of certain practical problems. Second has been the application of language-theory ideas, such as regular expressions and context-free grammars, in the design of software, such as compilers and text processors. Both of these developments have helped shape the organization of this book. USE OF THE BOOK Both authors have used Chapters 1 through 8 for a senior-level course, omitting only the material on inherent ambiguity in Chapter 4 and portions of Chapter 8. Chapters 7, 8, 12, and 13 form the nucleus of a course on computational complexity. An advanced course on language theory could be built around Chapters 2 through 7, 9 through 11, and 14. EXERCISES We use the convention that the most difficult problems are doubly starred, and problems of intermediate difficulty are identified by a single star. Exercises marked with an S have Downloaded fromv Ktunotes.in VI PREFACE solutions at the end of the chapter. We have not attempted to provide a solution manual, but have selected a few exercises whose solutions are particularly instructive. ACKNOWLEDGMENTS We would like to thank the following people for their perceptive comments and advice: Al Aho, Nissim Francez, Jon Goldstine, Juris Hartmanis, Dave Maier, Fred Springsteel, and Jacobo Valdes. The manuscript was expertly typed by Marie Olton and April Roberts at Cornell and Gerree Pecht at Princeton. Ithaca, New York J. E. H. Princeton, New Jersey J. D. U. March 1979 KTUNOTES.IN Downloaded from Ktunotes.in CONTENTS Chapter 1 Preliminaries 1.1 Strings, alphabets, and languages 1 1.2 Graphs and trees 2 1.3 Inductive proofs 4 1.4 Set notation 5 1.5 Relations 6 1.6KTUNOTES.INSynopsis of the book 8 Chapter 2 Finite Automata and Regular Expressions 2.1 Finite state systems 13 2.2 Basic definitions 16 2.3 Nondeterministic finite automata 19 2.4 Finite automata with (-moves 24 2.5 Regular expressions 28 2.6 Two-way finite automata 36 2.7 Finite automata with output 42 2.8 Applications of finite automata 45 Chapter 3 Properties of Regular Sets 3.1 The pumping lemma for regular sets 55 3.2 Closure properties of regular sets 58 3.3 Decision algorithms for regular sets 63 3.4 The Myhill-Nerode theorem and minimization of finite automata . 65 Downloaded fromvii Ktunotes.in Vlii CONTENTS Chapter 4 Context-Free Grammars 4.1 Motivation and introduction 77 4.2 Context-free grammars 79 4.3 Derivation trees 82 4.4 Simplification of context-free grammars 87 4.5 Chomsky normal form 92 4.6 Greibach normal form 94 4.7 The existence of inherently ambiguous context-free languages ... 99 Chapter 5 Pushdown Automata 5.1 Informal description 107 5.2 Definitions 108 5.3 Pushdown automata and context-free languages 114 Chapter 6 Properties of Context-Free Languages 6.1 The pumping lemma for CFL's 125 6.2 Closure properties of CFL's 130 6.3 Decision algorithms for CFL's 137 Chapter 7 Turing Machines 7.1 Introduction 146 7.2 The Turing machine model 147 7.3 Computable languages and functions 150 7.4 Techniques for Turing machine construction 153 7.5 Modifications of Turing machines 159 7.6KTUNOTES.INChurch's hypothesis 166 7.7 Turing machines as enumerators 167 7.8 Restricted Turing machines equivalent to the basic model 170 Chapter 8 Undecidability 8.1 Problems 177 8.2 Properties of recursive and recursively enumerable languages . 179 8.3 Universal Turing machines and an undecidable problem 181 8.4 Rice's theorem and some more undecidable problems 185 8.5 Undecidability of Post's correspondence problem 193 8.6 Valid and invalid computations of TM's: a tool for proving CFL problems undecidable 201 8.7 Greibach's theorem 205 8.8 Introduction to recursive function theory 207 8.9 Oracle computations 209 Chapter 9 The Chomsky Hierarchy 9.1 Regular grammars 217 9.2 Unrestricted grammars 220 Downloaded from Ktunotes.in CONTENTS IX 9.3 Context-sensitive languages 223 9.4 Relations between classes of languages 227 Chapter 10 Deterministic Context-Free Languages 10.1 Normal forms for DPDA's 234 10.2 Closure of DCFL's under complementation 235 10.3 Predicting machines 240 10.4 Additional closure properties of DCFL's 243 10.5 Decision properties of DCFL's 246 10.6 LR(0) grammars 248 10.7 LR(0) grammars and DPDA's 252 10.8 LR(k) grammars 260 Chapter 11 Closure Properties of Families of Languages 11.1 Trios and full trios 270 11.2 Generalized sequential machine mappings 272 11.3 Other closure properties of trios 276 11.4 Abstract families of languages 277 11.5 Independence of the AFL operations 279 11.6 Summary 279 Chapter 12 Computational Complexity Theory 12.1 Definitions 285 12.2 Linear speed-up, tape compression, and reductions in the number of tapes 288 12.3 Hierarchy theorems 295 12.4KTUNOTES.INRelations among complexity measures 300 12.5 Translational lemmas and nondeterministic hierarchies 302 12.6 Properties of general complexity measures: the gap, speedup, and union theorems 306 12.7 Axiomatic complexity theory 312 Chapter 13 Intractable Problems 13.1 Polynomial time and space 320 13.2 Some NP-complete problems 324 13.3 The class co-./T^ 341 13.4 PSPACE-complete problems 343 13.5 Complete problems for & and NSPACE(log n) 347 13.6 Some provably intractable problems 350 13.7 The 0> — jV'i? question for Turing machines with oracles: limits on our ability to tell whether & = c \'d? 362 Chapter 14 Highlights of Other Important Language Classes 14.1 Auxiliary pushdown automata 377 14.2 Stack automata 381 Downloaded from Ktunotes.in X CON ltNlb 14.3 Indexed languages 389 14.4 Developmental systems 390 Bibliography 396 Index 411 KTUNOTES.IN Downloaded from Ktunotes.in CHAPTER 1 PRELIMINARIES In this chapter we survey the principal mathematical ideas necessary for under- standing the material in this book. These concepts include graphs, trees, sets, relations, strings, abstract languages, and mathematical induction. We also pro- vide a brief introduction to, and motivation for, the entire work. The reader with a background in the mathematical subjects mentioned can skip to Section 1.6 for motivationalKTUNOTES.INremarks. 1.1 STRINGS, ALPHABETS, AND LANGUAGES A "symbol" is an abstract entity that we shall not define formally, just as "point" and "line" are not defined in geometry. Letters and digits are examples of frequently used symbols. A string (or word) is a finite sequence of symbols jux- taposed. For example, a, b, and c are symbols and abcb is a string. The length of a string w, denoted w , is the number of symbols composing the string. For exam- | | ple, abcb has length 4. The empty string, denoted by £, is the string consisting of zero symbols. Thus \e = 0. \ A prefix of a string is any number of leading symbols of that string, and a suffix is any number of trailing symbols. For example, string abc has prefixes £, a, ab, and abc; its suffixes are £, c, be, and abc. A prefix or suffix of a string, other than the string itself, is called a proper prefix or suffix. The concatenation of two strings is the string formed by writing the first, followed by the second, with no intervening space. For example, the concatena- tion of dog and house is doghouse. Juxtaposition is used as the concatenation operator. That is, if w and x are strings, then wx is the concatenation of these two Downloaded from1 Ktunotes.in 2 PRELIMINARIES strings.