Alberto Pettorossi Elements of Computability, Decidability, and Complexity Third Edition ARACNE Table of Contents Preface ................................................... ......................... 7 Chapter 1. TURING MACHINES ............................................. 9 1. Basic Definitions .................................. .............................. 9 2. Turing Machines ................................... ............................ 10 3. Techniques for Constructing Turing Machines . ........................ 18 3.1 Use of States to Store Information ................... ...................... 18 3.2 Use of Multiple Tracks ............................. ........................ 19 3.3 Test of String Equality ............................ ........................ 20 3.4 Shift of Strings .................................. .......................... 23 3.5 Use of Subroutines ................................ ........................ 23 4. Extensions of Turing Machines Which Do Not Increase Their Power ............ 24 4.1 Two-way Infinite Tape .............................. ....................... 24 4.2 Multitape Turing Machines ......................... ....................... 26 4.3 Multihead Turing Machines . ....................... 27 4.4 Off-line and On-line Turing Machines . ..................... 27 4.5 Bidimensional and Multidimensional Turing Machines . ................... 28 4.6 Nondeterministic versus Deterministic Turing Machines .................... 29 5. Restrictions to Turing Machines Which Do Not Diminish Their Power .......... 31 5.1 One Tape = One Tape + One Read-only Input Tape = Two Stacks . ....... 32 5.2 Two Stacks = Four Counters .......................... .................... 33 5.3 Four Counters = Two Counters ........................ .................... 35 5.4 Turing Machines with Three or Two States Only . ................. 37 5.5 Turing Machines with Three or Two Tape Symbols Only . .............. 38 6. Turing Machines as Language Generators ............... ....................... 38 7. Turing Computable Functions and Type 0 Languages . ................... 40 8. Multitape Turing Machines and Random Access Memory Machines ............. 42 9. Other Models of Computation ......................... ........................ 43 10. Church Thesis .................................... ............................. 52 Chapter 2. PARTIAL RECURSIVE FUNCTIONS ......................... 55 11. Preliminary Definitions ........................... ............................ 55 12. Primitive Recursive Functions . ............................ 57 13. Partial Recursive Functions ....................... ............................ 68 14. Partial Recursive Functions and Recursively EnumerableSets ................. 72 15. Recursive Enumerable Sets and Turing Computable Functions ................. 79 6 16. Partial Recursive Functions on Strings . .......................... 80 17. Partial Recursive Functions and Turing Computable Functions ................ 80 18. Subrecursive Classes of Partial Recursive Functions . ......................... 80 Chapter 3. DECIDABILITY .................................................. 87 19. Universal Turing Machines and Undecidability of the Halting Problem ......... 87 20. Rice Theorem ..................................... ............................ 94 21. Post Correspondence Problem ....................... .......................... 97 22. Decidability and Undecidability in Formal Languages . ....................... 98 22.1 Decidable Properties of Deterministic Context-free Languages ........... 103 22.2 Undecidable Properties of Deterministic Context-free Languages ........ 104 22.3 Summary of Decidability and Undecidability Results in Formal Languages 104 23. Decidability and Undecidability of Tiling Problems . ........................ 105 24. Arithmetical Hierarchy ........................... ............................ 108 Chapter 4. COMPUTATIONAL COMPLEXITY .......................... 113 25. Preliminary Definitions ........................... ........................... 113 26. P and NP Problems .................................. ....................... 115 27. Problem Reducibility ............................. ........................... 125 28. NP-complete Problems ............................. .......................... 128 28.1 Logarithmic-Space versus Polynomial-Time Reductions ................. 134 28.2 NP-Complete Problems and Strongly NP-Complete Problems ........... 135 28.3 NP Problems and co-NP Problems ..................... ................. 136 28.4 Complete Problems and Hard Problems . ................. 136 28.5 Approximation Algorithms ........................ ..................... 137 28.6 Stratification of Complexity Classes . ...................... 137 29. Computational Complexity Theory . ........................ 142 29.1 Deterministic Space and Deterministic Time Hierarchies ................ 142 29.2 Relationships among Complexity Classes . ................... 143 29.3 Nondeterministic Space and Nondeterministic Time Hierarchies ......... 144 29.4 Properties of General Complexity Measures . ................... 144 29.5 Computations with Oracles ........................ ..................... 145 30. Polynomial Hierarchy ............................. ........................... 145 31. Invariance with respect to the Computational Model . ..................... 147 32. Complexity Classes Based on Randomization and Probability ................ 153 33. Complexity Classes of Interactive Proofs . ......................... 158 34. Complexity Classes of Parallel Computations . ........................ 161 Index ................................................... ........................ 165 References ................................................... ................... 171 Preface These lecture notes are intended to introduce the reader to the basic notions of com- putability theory, decidability, and complexity. More information on these subjects can be found in classical books such as [6,8,15,16,33]. The results reported in these notes are taken from those books and in various parts we closely follow their style of presentation. The reader is encouraged to look at those books for improving his/her knowledge on these topics. Some parts of the chapter on complexity are taken from the lecture notes of a beautiful course given by Prof. Leslie Valiant at Edinburgh University, Scotland, in 1979. It was, indeed, a very stimulating and enjoyable course. For the notions of Predicate Calculus we have used in this book the reader may refer to [26]. I would like to thank Dr. Maurizio Proietti at IASI-CNR (Roma, Italy), my colleagues, and my students at the University of Roma Tor Vergata and, in particular, Michele Martone. They have been for me a source of continuous inspiration and enthusiasm. Finally, I would like to thank Dr. Gioacchino Onorati and Lorenzo Costantini of the Aracne Publishing Company for their helpful cooperation. Roma, July 2006 Preface to the Second Edition. Some typographical errors which were present in the first edition have been corrected. The chapters on Turing Machines and Decidability Theory have been revised. Roma, July 2007 Preface to the Third Edition. The chapter on Computational Complexity has been revised and some minor misprints have been corrected. Roma, October 2009 Alberto Pettorossi Department of Informatics, Systems, and Production University of Roma Tor Vergata Via del Politecnico 1, I-00133 Roma, Italy email: [email protected] URL: http://www.iasi.cnr.it/~adp Chapter 1 Turing Machines 1 Basic Definitions N = 0, 1, 2,... is the set of natural numbers. In what follows we will feel free to say { } ‘number’, instead of ‘natural number’. Given a set A and a number k 0, let Ak denote the set of all k-tuples of elements of A. Thus, by definition, we have that:≥ (i) A0 is the singleton made out of the 0-tuple (also called empty tuple or empty string or empty word and denoted by ε), that is, A0 =hi , {hi} (ii) A1 = x x A , and {h i| ∈ } (iii) Ak = x ,...,x x ,...,x A for any k > 1. {h 1 ki| 1 k ∈ } We identify x with x for each x A. Thus, A1 = A. Sometimes a tuple x ,...,x is h i ∈ h 1 ki also denoted by x1 ...xk. Let A∗ denote the set of all finite k-tuples of elements of A, with k 0, that is, ≥ A∗ = A0 A1 A2 . Ak . ∪ ∪ ∪ ∪ ∪ An element of A∗ is also called a string or a word. A∗ is also said to be the set of strings or words over A. The length of an element w of A∗, denoted w , is k iff w belongs to Ak, for any k 0. | | ≥ A set A is said to be denumerable (or countable) iff A has the cardinality of N or there exists a natural number n such that A has the same cardinality of n. The cardinality of a set A will be denoted by A . | | We have that A is denumerable iff A∗ is denumerable. The proof is based on the dove-tailing technique [32, Chapter 1]. Definition 1. [Irreflexive Total Order] Given a set A we say that a binary rela- tion < A A is an irreflexive total order (or a total order, for short) on A iff (i) < is a transitive⊆ × relation, and (ii) for any two elements a, b A we have that exactly one of ∈ these three cases occurs: either a<b, or a=b, or b<a. Let us consider a denumerable set A of symbols with a total order < among its elements. Then the canonical order, on the set of all words in A∗ is a binary relation, also denoted <, which is the subset of A∗ A∗ defined as follows. Given any two words w and × 1 w in A∗, we say that w canonically precedes w , and we write w <w , iff (i) either w 2 1 2 1 2 | 1| is smaller than w2 , (ii) or w1