Bibliography

In writing this book, I consulted [1–3] and revised some parts of the material from those books to make them accessible as parts of this book. The proof of the equiva- lence of context-free grammars and pushdown automata, as well as the proof of the NP completeness for Hamiltonian path problem and subset sum problem are based on the contents of [1]. Solutions to Problems 8.4 and 10.2 come from [2]. The construction of the universal Turing machine in Sect. 7.1 is from [3]. Furthermore, in discussing the general aspect of the theory of computation in computer science, I refer to [4–6]. Many other books other than those described above deal with the theory of com- putation. Regarding the broader introductory topics of computer science that in- clude the theory of computation as their theoretical core, several books have been published that are easy to understand, even for those who are not that familiar with computers. Biermann [8] is such an instructive book, although it is quite a large volume; Kawai [9] and Watanabe [10] are introductory books in this regard, but it is concisely organized for ease of reading; Wilf [11] and Iwama [12] are also introduc- tory books, but are somewhat focused on theoretical aspects; Inagaki [13] is another such book, which provides an account of computation as well as communication. For those readers who want to study further the topics of this book, but who need more advanced material or more comprehensive details, many useful books are available. Among them, Sipser [1] is a fascinating one which emphasizes throughout the intuitive bases underlying ideas. Papadimitriou [2] is remarkable for its depth of advanced arguments and its wide coverage of the topics. For that reason, this book is a must for readers who want to start research in these fields. Hopcroft et al. [14] is a comprehensive standard textbook that has been revised several times over the years. Its recent version emphasizes practical applications more than did the earlier versions. Savage [15] is another comprehensive book that particularly addresses the Boolean circuit model as well as the lower bounds of complexity in terms of cir- cuit size. Arora and Barak [16] describes recent achievements of complexity theory. Kasai [17] adopts a Turing machine as its computational model, expressing its be- havior in terms of a certain programming language introduced in the book. It gives arguments in detail, but succinctly. Koren [18] covers only the material described in Parts II and III of this book; it comprises lectures numbered 1 through 39 together with supplementary lectures, each being more or less self-contained.

A. Maruoka, Concise Guide to Computation Theory, 273 DOI 10.1007/978-0-85729-535-4, © Springer-Verlag London Limited 2011 274 Bibliography

Numerous books partially address the material contained in this book. Watan- abe [19] covers Parts III and IV; Kobayashi [20] covers Part IV; Kasai and Toda [21] covers Parts III and IV as well as parallel computation and approximate com- putation, which this book does not cover; Moriya [22] deals with Part II in detail; Tomita and Yokomori [23] and Yoneda et al. [24] explain the material of Parts II and III mainly using examples; Takeuti [25] specifically examines how to solve the P vs. NP problem; and Leeuwen [26] is a sort of encyclopedia of computer science, describing various topics written by authors who contributed to establishing their respective fields. Designing good algorithms is a fundamental common objective that is related somehow with every topic in this book. So many books have been published on how to design algorithms: Knuth [27, 28] give thorough arguments on designing algorithms; Hopcroft and Ullman [29], read extensively, has imparted a large influ- ence on the research on designing and analyzing algorithms; Ibaraki [30] is a good textbook on algorithms; and Hirata [31] is both concise and easy to read. The books, including others than those described above, that are referred to in this book are listed as follows. References

1. M. Sipser, Introduction to the Theory of Computation, Course Technology Ptr, Florence, 2005. 2. C. Papadimitriou, Computational Complexity, Addison-Wesley, Reading, 1994. 3. M. Minsky, Computation: Finite and Infinite Machines, Prentice-Hall, New York, 1967. 4. J. von Neumann, Theory of Self-Reproducing Automata, University of Illinois Press, Cham- paign, 1966. 5. L. Valiant, Circuit of the Mind, Oxford University Press, London, 1994. 6. J. Hartmanis, Observations About the Development of Theoretical Computer Science, Annals of the History of Computing, Vol. 3, No. 1, pp. 42–51, 1981. 7. G. Johnson, All Science Is Computer Science, The New York Times, March 25, 2001. 8. A. Biermann, Great Ideas in Computer Science, MIT Press, Cambridge, 1993. 9. S. Kawai, Computer Science, Tokyo University Press, Tokyo, 1995 (in Japanese). 10. O. Watanabe, Computer Science as a Culture, Saiensu-sha, Tokyo, 2001 (in Japanese). 11. H. Wilf, Algorithms and Complexity, AK Peters, Wellesley, 2002. 12. K. Iwama, Automaton-Languages and Computation Theory, Corona Publishing, San Antonio, 2003 (in Japanese). 13. Y. Inagaki, Communication and Computation, Iwanami-Shoten, Tokyo, 2003 (in Japanese). 14. J. Hopcroft, R. Motwani, and J. Ullman, Introduction to Automata Theory, Languages and Computation, Second Edition, Addison-Wesley, Reading, 2001. 15. J. Savage, Models of Computations, Addison-Wesley, Reading, 1998. 16. S. Arora and B. Barak, Computational Complexity—A Modern Approach, Cambridge Uni- versity Press, New York, 2009. 17. T. Kasai, The Theory of Computation, Kindai Kagaku sha, Tokyo, 1992 (in Japanese). 18. D. Koren, Automata and Computability, Springer, Berlin, 1997. 19. O. Watanabe, An Introduction to Computability and Computational Complexity, Kindai Ka- gaku sha, Tokyo, 1992 (in Japanese). 20. K. Kobayashi, Computational Complexity, Shokodo, Tokyo, 1988 (in Japanese). 21. T. Kasai and S. Toda, The Theory of Computation, Kyoritsu Shuppan, Tokyo, 1993 (in Japanese). 22. E. Moriya, Formal Languages and Automata, Saiensu-sha, Tokyo, 2001 (in Japanese). 23. E. Tomita and T. Yokomori, The Theory of Automaton and Language, Morikita Publishing, Tokyo, 1992 (in Japanese). 24. M. Yoneda, S. Hirose, N. Osato, and S. Okawa, The Foundation of the Theory of Automaton and Languages, Kindai Kagaku sha, Tokyo, 2003 (in Japanese). 25. G. Takeuti, P and NP, Nippon-Hyoron-Sha, Tokyo, 1996. 26. J. Leeuwen, Algorithms and Complexity, Volume A of Handbook of Computer Science, Else- vier, Amsterdam, 1990. 27. D. Knuth, The Art of Computer Programming, Vol. I, Addison-Wesley, Reading, 1973. 28. D. Knuth, The Art of Computer Programming, Vol. II, Addison-Wesley, Reading, 1981. 29. A. Hopcroft and J. Ullman, The Design and Analysis of Computer Algorithms, Addison- Wesley, Reading, 1974.

A. Maruoka, Concise Guide to Computation Theory, 275 DOI 10.1007/978-0-85729-535-4, © Springer-Verlag London Limited 2011 276 References

30. T. Ibaraki, Algorithms by C and Data Structures, Shokodo, Tokyo, 1999 (in Japanese). 31. T. Hirata, Algorithms and Data Structures, Morikita Publishing, Tokyo, 1990 (in Japanese). 32. M. Minsky, The Society of Mind, Simon & Schuster, New York, 1985. 33. N. Chomsky, Syntactic Structures, Mouton, The Hague, 1957. 34. V. Vazirani, Approximation Algorithms, Springer, Berlin, 2001. 35. L. Valiant, A Theory of the Learnable, Communications of the ACM, Vol. 15, No. 11, pp. 1134–1142, 1984. 36. A. Maruoka and E. Takimoto, Algorithmic Learning Theory, Encyclopedia of Computer Sci- ence and Technology, Vol. 45, Dekker, New York, 2002. 37. S. Arikawa, M. Sato, T. Sato, A. Maruoka, and Y. Kaneda, Structure and Development of Discovery Science, Proceedings of the Institute of Artificial Intelligence of Japan, Vol. 15, No. 4, pp. 595–607, 2000 (in Japanese). Index

A Chomsky normal form, 92, 93 A polynomial time verifier, 198 Church–Turing thesis, 6, 156, 157 A. Turing, 5 Circuit family, 205 Accept, 134, 136 Circuit model, 201 Accepting configuration, 136, 153 Circuit satisfiability problem, 217, 219 Accept state, 39, 49, 109, 134, 135, 141 CIRCUIT-SAT, 217–219 Adjacency matrix, 22, 30, 192 Class, 13 Alan Turing, 133, 162 Class of regular languages, 71 Algorithm, 3, 5, 28 Clause, 221 Alphabet, 15, 39, 49 Ambiguous grammar, 88 Clay Mathematics Institute, 7 AND gate, 201, 204 Clique, 199, 237, 266 Arc, 19 Closed, 74 Arithmetic expression, 86 Closed path, 21 Arrow, 19 Closed under operation, 74 Artificial automaton, 9 Closure property, 74, 75 Artificial intelligence, 10 Complement, 14 Artificial languages, 81 Computable in polynomial time, 174 Average-case analysis, 186 Computation, 4 Computation tree, 153 B Computational barrier, 4 Barber paradox, 167 Computational model, 3, 4 Bilaterally infinite tape, 147, 149 Concatenation, 15, 58 Block, 19 Configuration, 135 Boolean algebra, 22 Conjunction, 23 Boolean circuit, 7, 201, 204 Conjunctive normal form, 221 Boolean circuit family, 207 Connected graph, 21 Boolean formula, 23 Context, 82 Boolean function, 201, 204 Boolean operations, 22 Context-free grammar, 7, 8, 81, 82, 84, 89, 90, Boolean values, 22 96 Boolean variables, 22, 23 Context-free language, 81, 96 Brute-force search, 100, 191, 192 Context-sensitive grammar, 83, 89, 90, 96 Context-sensitive language, 96 C Contradiction, 170 Cartesian product, 14 Contraposition, 26 Cartesian product automaton, 75 Cover, 238, 267 Certificate, 185, 196 Cycle, 21 Chomsky hierarchy, 89, 96 CYK (Cocke–Younger–Kasami), 100

A. Maruoka, Concise Guide to Computation Theory, 277 DOI 10.1007/978-0-85729-535-4, © Springer-Verlag London Limited 2011 278 Index

D Formula satisfiability, 219 De Morgan’s law, 24, 31, 77, 203 Formula satisfiability problem, 217 Decidable, 137, 174 FORMULA-SAT, 217 Decide, 136 Function, 16 Decision problem, 17 Deep structure, 8 G Degree, 20, 194 Gate, 201, 204 Deletion of ε rules, 93 Generalization to a nondeterministic finite Deletion of long rules, 94 automaton, 44 Deletion of renaming rules, 94 Generalized Boolean circuit, 214 Derivation, 82, 85 Generalized De Morgan’s law, 242 Deterministic finite automaton, 39, 44, 48 Generalized finite circuit, 214 Deterministic Turing machine, 147, 152, 153 Generalized NFA, 67 DFA, 39, 48 Generalized nondeterministic finite automaton, Diagonalization, 169–171 68 Difference set, 13 Generates, 83, 85 Directed graph, 19 Generative grammar, 8 Discovery, 10 GNFA, 68 Disjunction, 23 Graph, 20 Distributive law, 23, 204 H Divided-by relation, 18 Halting problem, 6, 161, 167, 168, 174, 177 Domain, 16 Halting problem is undecidable, 169 DTM, 153 Hamiltonian graph, 195 Dynamic programming, 100 Hamiltonian path, 195 Hamiltonian path problem, 7, 191, 195, 227, E 230 ε-transition, 46, 51 HAMPATH, 230, 270 Edge, 19 Hard-to-solve relation, 171 Element, 13 Empty set, 13 I Empty string, 15 Indegree, 20 Empty tape halting problem, 180, 261 Indentation, 30 Entire current situation, 135 Independent set, 238, 267 Enumeration, 171 Induced, 21 Equivalence relation, 18, 241 Inductive definition, 59 Equivalent, 25, 60, 85 Infinite set, 13 Eulerian path, 193, 194 Infinite work tape, 134 Eulerian path problem, 191, 193, 194 Input, 18 Extended transition, 40 Input alphabet, 109, 134, 135 Extended transition function, 40 Inputs, 204 Instance for a problem, 18 F Intersection, 13 Factor, 86 Intractable, 227 Fan-in, 204 Intractable problems, 185 Fan-out, 204 Finite automaton, 35, 38, 39, 89 K Finite circuit, 206 k-clique, 199 Finite circuit family, 207 k-tape machine, 151 Finite function, 206 k-tuple, 15 Finite gate, 206 Finite input tape, 134 L Finite set, 13 L. Levin, 7 Formal definition, 39 L. Valiant, 10 Index 279

Labeled edge, 139 NP, 185, 197 Language, 15, 38 NP-complete, 227–229 Larger-than relation, 18 NP-completeness, 7 Leaf, 21 NTM, 153 Learning, 10 Leftmost derivation, 85 O Length, 15 One-to-one, 16 Length of a path, 21 Onto, 16 Lexicographical order, 155, 171 OR gate, 201, 204 Linear bounded automaton, 89, 90 Oracle, 172 Literal, 221 Oracle tape, 172 Long rule, 93 Oracle Turing machine, 172 Loop, 134 Outdegree, 20 Output, 18 M Machine, 5 Output gate, 201 Machine condition region, 162, 163, 165 Machine description region, 162, 163 P Mapping, 16 P vs. NP problem, 7, 228 Mapping reducibility, 173 Pair, 15 Mapping reducible, 172 Parse tree, 82, 84 Match, 175 Partition, 18, 241 Mathematical induction, 28 Path, 21 mcnf-formula, 223 PCP, 177 Mechanical procedure, 3, 5 Phase structured grammars, 9 Membership problem, 17, 100 Phrase structure grammar, 89, 90, 96 Membership problem for context-free Phrase structure language, 96 languages, 100 Pigeonhole principle, 57, 74 MEN, 262 Polynomial time, 190 Model, 10 Polynomial time reducibility, 173 Modified Post correspondence problem, 176 Polynomial time reducible, 172 MPCP, 176, 177 Polynomial time Turing machine, 198 MSAT, 223 Polynomially equivalent, 189 Multi-set, 15 Positive result, 171 Multiple tape, 147 Post correspondence problem, 161, 174, 177, Multitape Turing machine, 149 180, 181 Power set, 14 N Precedence order, 59 N. Chomsky, 8 Predicative logic, 22 Natural automaton, 9 Prefix, 104 Natural language, 81 Problem, 3, 10, 16 Negation, 23 Program, 10 Negative result, 171 Proof by contradiction, 26, 27 Neurocomputer, 10 NFA, 48 Proof by contraposition, 26, 27 Node, 19 Proper subset, 13 NODE-COVER, 267 Proposition, 24 Nondeterministic, 198 Propositional logic, 22 Nondeterministic finite automaton, 44, 45, 49 Pumping lemma, 73, 81 Nondeterministic Turing machine, 147, 152, Pumping lemma for context-free languages, 98 153 Pushdown automaton, 7, 89, 107, 109 Nonterminal symbol, 83 Nonterminal, 84, 91 Q NOT gate, 201, 204 Quantum computer, 10 280 Index

R State transition, 111 Range, 16 State transition diagram, 38, 39 REACH, 192 State transition function, 39 Reachability problem, 28, 180, 181, 191, 192 State transition table, 39 Read-only head, 134 State, 49, 109, 135 Read-write head, 134 Stored-program, 9 Recognizable, 137 Stored-program computer, 161, 162 Recursive function, 8 String, 15 Recursively enumerable, 96 Reducibility, 171, 174 Subgraph, 21 Reduction, 171, 180 Subset, 13 Reflexive law, 18, 31 Subset sum problem, 227, 235 Regular expression, 57, 58, 258 SUBSET-SUM, 235, 236 Regular grammar, 89, 91, 96 Substitution rule, 82, 84, 91 Regular language, 71, 96 Substring, 15 Reject, 134 Surface structure, 8 Reject state, 134, 135, 141 Symmetric, 22 Rejecting configuration, 153 Symmetric law, 18, 31 Relation, 18 Renaming rule, 93 Repeat condition, 72, 74 T Rewriting rule, 8, 81 2SAT, 238 Robust, 8 3CLIQUE, 262 Root, 21 3SAT, 224, 229, 230 Rooted tree, 21 Tape alphabet, 134, 135 Rule, 82, 84 Tape head, 133 Tape region, 162 S Terminal, 84, 91 S. Arikawa, 10 Terms, 86 S. Cook, 7 Three-way model, 158 S. Kleene, 8 Time complexity, 6, 185, 186, 189 SAT, 217, 219 Time complexity of a problem, 189 Satisfiability problem, 217, 227, 229 t(n) Satisfiable, 217 time Turing machine, 186 Satisfiable Boolean circuit, 217 Tractable, 227 Sentential form, 85 Tractable problem, 185 Set, 13 Transition function, 39, 49, 109, 133, 135, 152 Short certificate, 197 Transitive closure, 21 Short string, 196 Transitive law, 18, 31, 173 Simple path, 21 Traveling salesman problem, 237, 266 Single tape, 147 Tree, 21 Single-tape machine, 151 Truth table, 202 Single-tape Turing machine, 149 TSP, 266 Size, 13, 205 Turing machine, 5, 89, 133, 135 Size complexity, 205 Turing machine as a transformer, 134 Solve quickly, 196 Turing machine with a single tape, 134 Stack, 7, 107 Turing machines, 9 Stack alphabet, 107, 109 Stack head, 107 Turing reducibility, 172 Start configuration, 136, 153 Turing reducible, 172 Start state, 39, 109, 135, 141 Turing test, 10 Start symbol, 83, 84 Turing-decidable, 137 State, 39 Turing-recognizable, 96, 137 State diagram, 38, 39, 139 Two-way model, 158 Index 281

U Verifier, 196, 197 UHAMPATH, 238, 270 Vertex, 19 Unambiguous, 88 Von Neumann, 9 Undecidable, 167, 174 Undirected graph, 20 W Unilaterally infinite tape, 147, 149 Well-nested string, 186 Union, 13, 58 Well-nested string of parentheses, 85 Universal Turing machine, 161, 163 Worst-case analysis, 186 V Variable, 84 Y Venn diagram, 14 YES/NO problem, 17