sign in sign up
<<
Home , Codomain

Contents

1 Post correspondence problem 1 1.1 Definition of the problem ...... 1 1.1.1 Alternative definition ...... 1 1.2 Example instances of the problem ...... 1 1.2.1 Example 1 ...... 1 1.2.2 Example 2 ...... 1 1.3 Proof sketch of undecidability ...... 2 1.4 Variants ...... 2 1.5 References ...... 3 1.6 External links ...... 3

2 Halting problem 4 2.1 Background ...... 4 2.2 Importance and consequences ...... 4 2.3 Representation as a ...... 5 2.4 Sketch of proof ...... 5 2.5 Proof as a corollary of the uncomputability of Kolmogorov complexity ...... 6 2.6 Common pitfalls ...... 6 2.7 Formalization ...... 7 2.8 Relationship with Gödel’s incompleteness ...... 7 2.9 Variants of the halting problem ...... 7 2.9.1 Halting on all inputs ...... 8 2.9.2 Recognizing partial solutions ...... 8 2.9.3 Generalized to oracle machines ...... 8 2.10 History ...... 8 2.11 Avoiding the halting problem ...... 9 2.12 See also ...... 9 2.13 Notes ...... 9 2.14 References ...... 9 2.15 External links ...... 11 2.16 Text and sources, contributors, and licenses ...... 12 2.16.1 Text ...... 12 2.16.2 Images ...... 12

i ii CONTENTS

2.16.3 Content license ...... 12 Chapter 1

Post correspondence problem

Not to be confused with the other Post’s problem on the existence of incomparable r.e. Turing degrees. Not to be confused with PCP . g(w) = h(w)

The Post correspondence problem is an undecidable 1.2 Example instances of the prob- decision problem that was introduced by Emil Post in 1946.[1] Because it is simpler than the halting problem lem and the it is often used in proofs of undecidability. 1.2.1 Example 1

Consider the following two lists: 1.1 Definition of the problem A solution to this problem would be the (3, 2, 3, 1), because The input of the problem consists of two finite lists α1, . . . , αN and β1, . . . , βN of words over some alphabet A having at least two symbols. A solution to this prob- α3α2α3α1 = bba+ab+bba+a = bbaabbbaa = bb+aa+bb+baa = β3β2β3β1. lem is a sequence of indices (ik)1≤k≤K with K ≥ 1 and 1 ≤ ik ≤ N for all k , such that Furthermore, since (3, 2, 3, 1) is a solution, so are all of its “repetitions”, such as (3, 2, 3, 1, 3, 2, 3, 1), etc.; that is, when a solution exists, there are infinitely many solutions of this repetitive kind. αi1 . . . αiK = βi1 . . . βiK . However, if the two lists had consisted of only α2, α3 and The decision problem then is to decide whether such a β2, β3 from those sets, then there would have been no solution exists or not. solution (the last letter of any such α string is not the same as the letter before it, whereas β only constructs pairs of 1.1.1 Alternative definition the same letter). A convenient way to view an instance of a Post correspon- In the above definition, a solution can be seen as a dence problem is as a collection of blocks of the form { } nonempty word on the alphabet 1,...,N , and each there being an unlimited supply of each type of block. of the two given lists of words on alphabet A can be Thus the above example is viewed as seen as defining a that maps words on {1,...,N} to words on A : where the solver has an endless supply of each of these three block types. A solution corresponds to some way of laying blocks next to each other so that the string in the top cells corresponds to the string in the bottom cells. g :(i , . . . , i ) 7→ α . . . α 1 K i1 iK Then the solution to the above example corresponds to: 7→ h :(i1, . . . , iK ) βi1 . . . βiK . This gives rise to an equivalent alternative definition often 1.2.2 Example 2 found in the literature, according to which any two homo- g, h with a common domain and a common Again using blocks to represent an instance of the prob- codomain form an instance of the Post correspondence lem, the following is an example that has infinitely many problem, which now asks whether there exists a nonempty solutions in addition to the kind obtained by merely “re- word w in the domain such that peating” a solution.

1 2 CHAPTER 1. POST CORRESPONDENCE PROBLEM

In this instance, every sequence of the form (1, 2, 2, . . ., the finite state changes, and what happens to the surround- 2, 3) is a solution (in addition to all their repetitions): ing symbols. For example, here the tape head is over a 0 in state 4, and then writes a 1 and moves right, changing to state 7: 1.3 Proof sketch of undecidability Finally, when the top reaches an accepting state, the bot- tom needs a chance to finally catch up to complete the The most common proof for the undecidability of PCP match. To allow this, we extend the computation so that describes an instance of PCP that can simulate the com- once an accepting state is reached, each subsequent ma- putation of a Turing machine on a particular input. A chine step will cause a near the tape head to van- match will occur if and only if the input would be ac- ish, one at a time, until none remain. If qf is an accepting cepted by the Turing machine. Because deciding if a Tur- state, we can represent this with the following transition ing machine will accept an input is a basic undecidable blocks, where a is a tape alphabet symbol: problem, PCP cannot be decidable either. The following There are a number of details to work out, such as dealing discussion is based on Michael Sipser's textbook Intro- with boundaries between states, making sure that our ini- [2] duction to the Theory of Computation. tial tile goes first in the match, and so on, but this shows In more detail, the idea is that the string along the top the general idea of how a static tile puzzle can simulate a and bottom will be a computation history of the Turing Turing machine computation. machine’s computation. This means it will list a string de- The previous example scribing the initial state, followed by a string describing the next state, and so on until it ends with a string describ- q0101#1q401#11q21#1q810. ing an accepting state. The state strings are separated by is represented as the following solution to the Post corre- some separator symbol (usually written #). According to spondence problem: the definition of a Turing machine, the full state of the machine consists of three parts: 1.4 Variants • The current contents of the tape. Many variants of PCP have been considered. One reason • The current state of the finite state machine which is that, when one tries to prove undecidability of some operates the tape head. new problem by reducing from PCP, it often happens that • The current position of the tape head on the tape. the first reduction one finds is not from PCP itself but from an apparently weaker version. Although the tape has infinitely many cells, only some fi- nite prefix of these will be non-blank. We write these • The problem may be phrased in terms of monoid ∗ down as part of our state. To describe the state of the fi- morphisms f, g from the free monoid B to the free ∗ nite control, we create new symbols, labelled q1 through monoid A where B is of size n. The problem is to qk, for each of the finite state machine’s k states. We determine whether there is a word w in B+ such that insert the correct symbol into the string describing the f(w) = g(w).[3] tape’s contents at the position of the tape head, thereby • The condition that the alphabet A have at least two indicating both the tape head’s position and the current symbols is required since the problem is decidable state of the finite control. For the alphabet {0,1}, a typi- if A has only one symbol. cal state might look something like: • A simple variant is to fix n, the number of tiles. This 101101110q700110. problem is decidable if n ≤ 2, but remains undecid- A simple computation history would then look something able for n ≥ 5. It is unknown whether the problem like this: is decidable for 3 ≤ n ≤ 4.[4] q 101#1q 01#11q 1#1q 10. 0 4 2 8 • The circular Post correspondence problem asks We start out with this block, where x is the input string whether indexes i1, i2,... can be found such that and q0 is the start state: ··· ··· αi1 αik and βi1 βik are conjugate words, i.e., The top starts out “lagging” the bottom by one state, and they are equal modulo rotation. This variant is [5] keeps this lag until the very end stage. Next, for each undecidable. symbol a in the tape alphabet, as well as #, we have a • One of the most important variants of PCP is the “copy” block, which copies it unmodified from one state bounded Post correspondence problem, which to the next: asks if we can find a match using no more than k We also have a block for each position transition the ma- tiles, including repeated tiles. A brute force search chine can make, showing how the tape head moves, how solves the problem in time O(2k), but this may be 1.6. EXTERNAL LINKS 3

difficult to improve upon, since the problem is NP- International Symposium on Theoretical Aspects of Com- complete.[6] Unlike some NP-complete problems puter Science (STACS 2015). STACS 2015. Schloss like the boolean satisfiability problem, a small vari- Dagstuhl–Leibniz-Zentrum fuer Informatik. pp. 649– ation of the bounded problem was also shown to be 661. doi:10.4230/LIPIcs.STACS.2015.649. complete for RNP, which means that it remains hard [5] K. Ruohonen (1983). “On some variants of Post’s corre- even if the inputs are chosen at random (it is hard on spondence problem”. Acta Informatica (Springer) 19 (4): [7] average over uniformly distributed inputs). 357–367. doi:10.1007/BF00290732.

• Another variant of PCP is called the marked Post [6] Michael R. Garey; David S. Johnson (1979). Computers Correspondence Problem, in which each ui must and Intractability: A Guide to the Theory of NP- begin with a different symbol, and each vi must also Completeness. W.H. Freeman. p. 228. ISBN 0-7167- begin with a different symbol. Halava, Hirvensalo, 1045-5. and de Wolf showed that this variation is decidable [7] Y. Gurevich (1991). “Average case completeness”. J. in exponential time. Moreover, they showed that Comp. Sys. Sci. (Elsevier Science) 42 (3): 346–398. if this requirement is slightly loosened so that only doi:10.1016/0022-0000(91)90007-R. one of the first two characters need to differ (the so-called 2-marked Post Correspondence Problem), [8] V. Halava; M. Hirvensalo; R. de Wolf (2001). “Marked the problem becomes undecidable again.[8] PCP is decidable”. Theor. Comp. Sci. (Elsevier Science) 255: 193–204. doi:10.1016/S0304-3975(99)00163-2. • The Post Embedding Problem is another vari- [9] P. Chambart; Ph. Schnoebelen (2007). “Post embed- ant where one looks for indexes i1, i2,... such that ding problem is not primitive recursive, with applica- ··· ··· αi1 αik is a (scattered) subword of βi1 βik . tions to channel systems”. Lecture Notes in Computer This variant is easily decidable since, when some Science. Lecture Notes in Computer Science (Springer) solutions exist, in particular a length-one solution 4855: 265–276. doi:10.1007/978-3-540-77050-3_22. exists. More interesting is the Regular Post Em- ISBN 978-3-540-77049-7. bedding Problem, a further variant where one looks [10] Paul C. Bell; Igor Potapov (2010). “On the Undecid- for solutions that belong to a given regular language ability of the Identity Correspondence Problem and its (submitted, e.g., under the form of a regular ex- Applications for Word and Semigroups”. In- pression on the set {1,...,N} ). The Regular Post ternational Journal of Foundations of Computer Science Embedding Problem is still decidable but, because (World Scientific) 21.6: 963–978. arXiv:0902.1975. of the added regular constraint, it has a very high doi:10.1142/S0129054110007660. complexity that dominates every multiply recursive .[9] 1.6 External links • The Identity Correspondence Problem (ICP) asks whether a finite set of pairs of words (over a group • alphabet) can generate an identity pair by a sequence Eitan M. Gurari. An Introduction to the Theory of concatenations. The problem is undecidable and of Computation, Chapter 4, Post’s Correspondence equivalent to the following Group Problem: is the Problem. A proof of the undecidability of PCP semigroup generated by a finite set of pairs of words based on Chomsky type-0 grammars. [10] (over a group alphabet) a group. • Online PHP Based PCP Solver

• PCP AT HOME 1.5 References • PCP - a nice problem

[1] E. L. Post (1946). “A variant of a recursively unsolvable problem” (PDF). Bull. Amer. Math. Soc 52.

[2] Michael Sipser (2005). “A Simple Undecidable Prob- lem”. Introduction to the Theory of Computation (2nd ed.). Thomson Course Technology. pp. 199–205. ISBN 0- 534-95097-3.

[3] Salomaa, Arto (1981). Jewels of The- ory. Pitman Publishing. pp. 74–75. ISBN 0-273-08522- 0. Zbl 0487.68064.

[4] T. Neary (2015). “Undecidability in Binary Tag Systems and the Post Correspondence Problem for Five Pairs of Words”. In Ernst W. Mayr and Nicolas Ollinger. 32nd Chapter 2

Halting problem

In , the halting problem is the prob- One approach to the problem might be to run the program lem of determining, from a description of an arbitrary for some number of steps and check if it halts. But if the computer program and an input, whether the program program does not halt, it is unknown whether the program will finish running or continue to run forever. will eventually halt or run forever. Alan Turing proved in 1936 that a general algorithm to Turing proved no algorithm exists that always correctly solve the halting problem for all possible program-input decides whether, for a given arbitrary program and input, pairs cannot exist. A key part of the proof was a math- the program halts when run with that input; the essence ematical definition of a computer and program, which of Turing’s proof is that any such algorithm can be made became known as a Turing machine; the halting problem to contradict itself, and therefore cannot be correct. is undecidable over Turing machines. It is one of the first examples of a decision problem. Jack Copeland (2004) attributes the term halting problem to Martin Davis.[1] 2.2 Importance and consequences

The halting problem is historically important because it 2.1 Background was one of the first problems to be proved undecidable. (Turing’s proof went to press in May 1936, whereas Alonzo Church's proof of the undecidability of a prob- The halting problem is a decision problem about prop- lem in the lambda calculus had already been published erties of computer programs on a fixed Turing-complete in April 1936.) Subsequently, many other undecidable model of computation, i.e., all programs that can be writ- problems have been described; the typical method of ten in some given programming language that is general proving a problem to be undecidable is with the tech- enough to be equivalent to a Turing machine. The prob- nique of reduction. To do this, it is sufficient to show lem is to determine, given a program and an input to the that if a solution to the new problem were found, it could program, whether the program will eventually halt when be used to decide an undecidable problem by transform- run with that input. In this abstract framework, there are ing instances of the undecidable problem into instances of no resource limitations on the amount of memory or time the new problem. Since we already know that no method required for the program’s execution; it can take arbitrar- can decide the old problem, no method can decide the ily long, and use arbitrarily as much storage space, before new problem either. Often the new problem is reduced to halting. The question is simply whether the given pro- solving the halting problem. (Note: the same technique gram will ever halt on a particular input. is used to demonstrate that a problem is NP complete, For example, in pseudocode, the program only in this case, rather than demonstrating that there is no solution, it demonstrates there is no polynomial time while (true) continue solution, assuming P ≠ NP). For example, one such consequence of the halting prob- does not halt; rather, it goes on forever in an infinite loop. lem’s undecidability is that there cannot be a general On the other hand, the program algorithm that decides whether a given statement about natural numbers is true or not. The reason for this is print “Hello, world!" that the stating that a certain program will halt given a certain input can be converted into an equivalent statement about natural numbers. If we had an algorithm does halt. that could find the of every statement about While deciding whether these programs halt is simple, natural numbers, it could certainly find the truth value more complex programs prove problematic. of this one; but that would determine whether the orig-

4 2.3. REPRESENTATION AS A SET 5 inal program halts, which is impossible, since the halting in the long run, elude simulation by a Turing machine, problem is undecidable. and in particular whether any such hypothetical process Rice’s theorem generalizes the theorem that the halting could usefully be harnessed in the form of a calculating problem is unsolvable. It states that for any non-trivial machine (a hypercomputer) that could solve the halting property, there is no general decision procedure that, for problem for a Turing machine amongst other things. It is all programs, decides whether the imple- also an open question whether any such unknown phys- mented by the input program has that property. (A partial ical processes are involved in the working of the human function is a function which may not always produce a re- brain, and whether humans can solve the halting problem (Copeland 2004, p. 15). sult, and so is used to model programs, which can either produce results or fail to halt.) For example, the prop- erty “halt for the input 0” is undecidable. Here, “non- trivial” means that the set of partial functions that satisfy 2.3 Representation as a set the property is neither the nor the set of all par- tial functions. For example, “halts or fails to halt on input The conventional representation of decision problems is 0” is clearly true of all partial functions, so it is a trivial the set of objects possessing the property in question. The property, and can be decided by an algorithm that sim- halting set ply reports “true.” Also, note that this theorem holds only for properties of the partial function implemented by the K := { (i, x) | program i halts when run on input program; Rice’s Theorem does not apply to properties of x} the program itself. For example, “halt on input 0 within 100 steps” is not a property of the partial function that is represents the halting problem. implemented by the program—it is a property of the pro- gram implementing the partial function and is very much This set is recursively enumerable, which means there is decidable. a that lists all of the pairs (i, x) it contains.[2] However, the complement of this set is not Gregory Chaitin has defined a halting probability, repre- recursively enumerable.[2] sented by the symbol Ω, a type of real number that in- formally is said to represent the probability that a ran- There are many equivalent formulations of the halting domly produced program halts. These numbers have the problem; any set whose Turing degree equals that of the same Turing degree as the halting problem. It is a normal halting problem is such a formulation. Examples of such and transcendental number which can be defined but can- sets include: not be completely computed. This means one can prove that there is no algorithm which produces the digits of • { i | program i eventually halts when run with input Ω, although its first few digits can be calculated in simple 0 } cases. • { i | there is an input x such that program i eventually While Turing’s proof shows that there can be no general halts when run with input x }. method or algorithm to determine whether algorithms halt, individual instances of that problem may very well be susceptible to attack. Given a specific algorithm, one 2.4 Sketch of proof can often show that it must halt for any input, and in fact computer scientists often do just that as part of a correctness proof. But each proof has to be developed The proof shows there is no total computable function specifically for the algorithm at hand; there is no mechan- that decides whether an arbitrary program i halts on arbi- ical, general way to determine whether algorithms on a trary input x; that is, the following function h is not com- Turing machine halt. However, there are some heuristics putable (Penrose 1990, p. 57–63): that can be used in an automated fashion to attempt to construct a proof, which succeed frequently on typical { programs. This field of research is known as automated 1 if program i input on halts x, h(i, x) = termination analysis. 0 otherwise. Since the negative answer to the halting problem shows Here program i refers to the i th program in an that there are problems that cannot be solved by a Tur- of all the programs of a fixed Turing- ing machine, the Church–Turing thesis limits what can be complete model of computation. accomplished by any machine that implements effective methods. However, not all machines conceivable to hu- Possible values for a total computable function f arranged in man imagination are subject to the Church–Turing thesis a 2D array. The orange cells are the diagonal. The values of (e.g. oracle machines). It is an open question whether f(i,i) and g(i) are shown at the bottom; U indicates that the there can be actual deterministic physical processes that, function g is undefined for a particular input value. 6 CHAPTER 2. HALTING PROBLEM

The proof proceeds by directly establishing that every to- using the main diagonal of this array. If the array has a tal computable function with two differs from 0 at position (i,i), then g(i) is 0. Otherwise, g(i) is unde- the required function h. To this end, given any total com- fined. The contradiction comes from the fact that there putable binary function f, the following partial function g is some column e of the array corresponding to g itself. is also computable by some program e: Now assume f was the halting function h, if g(e) is defined ( g(e) = 0 in this case ), g(e) halts so f(e,e) = 1. But g(e) { = 0 only when f(e,e) = 0, contradicting f(e,e) = 1. Sim- 0 iff(i, i) = 0, ilarly, if g(e) is not defined, then halting function f(e,e) g(i) = undefined otherwise. = 0, which leads to g(e) = 0 under g's construction. This contradicts the assumption that g(e) not being defined. In The verification that g is computable relies on the follow- both cases contradiction arises. Therefore any arbitrary ing constructs (or their equivalents): computable function f cannot be the halting function h.

• computable subprograms (the program that com- putes f is a subprogram in program e), 2.5 Proof as a corollary of the un- • duplication of values (program e computes the in- computability of Kolmogorov puts i,i for f from the input i for g), complexity • conditional branching (program e selects between two results depending on the value it computes for The undecidability of the halting problem also follows f(i,i)), from the fact that Kolmogorov complexity is not com- • not producing a defined result (for example, by loop- putable. If the halting problem were decidable, it would ing forever), be possible to construct a program that generated pro- grams of increasing length, running those that halt and • returning a value of 0. comparing their final outputs with a string parameter un- til one matched (which must happen eventually, as any The following pseudocode illustrates a straightforward string can be generated by a program that contains it as way to compute g: data and just lists it); the length of the matching gener- ated program would then be the Kolmogorov complexity procedure compute_g(i): if f(i,i) == 0 then return 0 else of the parameter, as the terminating generated program loop forever must be the shortest (or shortest equal) such program.[3]

Because g is partial computable, there must be a program e that computes g, by the assumption that the model of computation is Turing-complete. This program is one of 2.6 Common pitfalls all the programs on which the halting function h is de- fined. The next step of the proof shows that h(e,e) will The difficulty in the halting problem lies in the require- not have the same value as f(e,e). ment that the decision procedure must work for all pro- grams and inputs. A particular program either halts on a It follows from the definition of g that exactly one of the given input or does not halt. Consider one algorithm that following two cases must hold: always answers “halts” and another that always answers “doesn't halt”. For any specific program and input, one • f(e,e) = 0 and so g(e) = 0. In this case h(e,e) = 1, of these two algorithms answers correctly, even though because program e halts on input e. nobody may know which one. • f(e,e) ≠ 0 and so g(e) is undefined. In this case h(e,e) There are programs (interpreters) that simulate the exe- = 0, because program e does not halt on input e. cution of whatever source code they are given. Such pro- grams can demonstrate that a program does halt if this In either case, f cannot be the same function as h. Be- is the case: the interpreter itself will eventually halt its cause f was an arbitrary total computable function with simulation, which shows that the original program halted. two arguments, all such functions must differ from h. However, an interpreter will not halt if its input program This proof is analogous to Cantor’s diagonal . does not halt, so this approach cannot solve the halt- One may visualize a two-dimensional array with one col- ing problem as stated. It does not successfully answer umn and one row for each natural number, as indicated “doesn't halt” for programs that do not halt. in the table above. The value of f(i,j) is placed at column The halting problem is theoretically decidable for linear i, row j. Because f is assumed to be a total computable bounded automata (LBAs) or deterministic machines function, any of the array can be calculated us- with finite memory. A machine with finite memory has ing f. The construction of the function g can be visualized a finite number of states, and thus any deterministic pro- 2.8. RELATIONSHIP WITH GÖDEL’S INCOMPLETENESS THEOREMS 7 gram on it must eventually either halt or repeat a previous alphabet with n characters can also be mapped to num- state: bers by interpreting them as numbers in an n-ary numeral system. ...any finite-state machine, if left completely to itself, will fall eventually into a perfectly peri- odic repetitive pattern. The duration of this re- 2.8 Relationship with Gödel’s in- peating pattern cannot exceed the number of internal states of the machine... (italics in orig- completeness theorems inal, Minsky 1967, p. 24) The concepts raised by Gödel’s incompleteness theorems Minsky warns us, however, that machines such as com- are very similar to those raised by the halting problem, puters with e.g., a million small parts, each with two and the proofs are quite similar. In fact, a weaker form of states, will have at least 21,000,000 possible states: the First Incompleteness Theorem is an easy consequence of the undecidability of the halting problem. This weaker form differs from the standard statement of the incom- This is a 1 followed by about three hundred pleteness theorem by asserting that a complete, consistent thousand zeroes ... Even if such a machine and sound axiomatization of all statements about natural were to operate at the frequencies of cosmic numbers is unachievable. The “sound” part is the weak- rays, the aeons of galactic evolution would be ening: it means that we require the in as nothing compared to the time of a journey question to prove only true statements about natural num- through such a cycle (Minsky 1967 p. 25): bers. The more general statement of the incompleteness theorems does not require a soundness assumption of this Minsky exhorts the reader to be suspicious—although a kind. machine may be finite, and finite automata “have a num- ber of theoretical limitations": The weaker form of the theorem can be proven from the undecidability of the halting problem as follows. Assume that we have a consistent and complete axiomatization ...the magnitudes involved should lead one to of all true first-order logic statements about natural num- suspect that theorems and arguments based bers. Then we can build an algorithm that enumerates all chiefly on the mere finiteness [of] the state dia- these statements. This means that there is an algorithm gram may not carry a great deal of significance. N(n) that, given a natural number n, computes a true first- (Minsky p. 25) order logic statement about natural numbers such that, for all the true statements, there is at least one n such that It can also be decided automatically whether a nondeter- N(n) yields that statement. Now suppose we want to de- ministic machine with finite memory halts on none, some, cide whether the algorithm with representation a halts on or all of the possible of nondeterministic deci- input i. By using Kleene’s T predicate, we can express sions, by enumerating states after each possible decision. the statement "a halts on input i" as a statement H(a, i) in the language of arithmetic. Since the axiomatization is complete it follows that either there is an n such that 2.7 Formalization N(n) = H(a, i) or there is an n' such that N(n') = ¬ H(a, i). So if we iterate over all n until we either find H(a, In his original proof Turing formalized the concept of i) or its negation, we will always halt. This means that algorithm by introducing Turing machines. However, the this gives us an algorithm to decide the halting problem. result is in no way specific to them; it applies equally Since we know that there cannot be such an algorithm, it to any other model of computation that is equivalent follows that the assumption that there is a consistent and in its computational power to Turing machines, such complete axiomatization of all true first-order logic state- as Markov algorithms, Lambda calculus, Post systems, ments about natural numbers must be false. register machines, or tag systems. What is important is that the formalization allows a straightforward mapping of algorithms to some data type 2.9 Variants of the halting problem that the algorithm can operate upon. For example, if the formalism lets algorithms define functions over strings Many variants of the halting problem can be found in (such as Turing machines) then there should be a mapping Computability textbooks (e.g., Sipser 2006, Davis 1958, of these algorithms to strings, and if the formalism lets al- Minsky 1967, Hopcroft and Ullman 1979, Börger 1989). gorithms define functions over natural numbers (such as Typically their undecidability follows by reduction from computable functions) then there should be a mapping of the standard halting problem. However, some of them algorithms to natural numbers. The mapping to strings have a higher degree of unsolvability. The next two ex- is usually the most straightforward, but strings over an amples are typical. 8 CHAPTER 2. HALTING PROBLEM

2.9.1 Halting on all inputs “Of these, the second was that of proving the con- sistency of the 'Peano ' on which, as he The universal halting problem, also known (in had shown, the rigour of mathematics depended”. theory) as totality, is the problem of determining, whether (Hodges p. 83, Davis’ commentary in Davis, 1965, a given computer program will halt for every input (the p. 108) name totality comes from the equivalent question of • whether the computed function is total). This problem is 1920–1921: Emil Post explores the halting prob- not only undecidable, as the halting problem, but highly lem for tag systems, regarding it as a candidate for undecidable. In terms of the Arithmetical hierarchy, it is unsolvability. (Absolutely unsolvable problems and 0 [4] relatively undecidable – account of an Π2 -complete. This means, in particular, that it cannot be decided even with an oracle for the halting problem. anticipation, in Davis, 1965, pp. 340–433.) Its un- solvability was not established until much later, by Marvin Minsky (1967). 2.9.2 Recognizing partial solutions • 1928: Hilbert recasts his 'Second Problem' at the Bologna International Congress. (Reid pp. 188– There are many programs that, for some inputs, return 189) Hodges claims he posed three questions: i.e. a correct answer to the halting problem, while for other #1: Was mathematics complete? #2: Was math- inputs they do not return an answer at all. However the ematics consistent? #3: Was mathematics decid- problem ″given program p, is it a partial halting solver″ able? (Hodges p. 91). The third question is known (in the sense described) is at least as hard as the halt- as the Entscheidungsproblem (Decision Problem). ing problem. To see this, assume that there is an algo- (Hodges p. 91, Penrose p. 34) rithm PHSR (″partial halting solver recognizer″) to do that. Then it can be used to solve the halting problem, • 1930: Kurt Gödel announces a proof as an answer to as follows: To test whether input program x halts on y, the first two of Hilbert’s 1928 questions [cf Reid p. construct a program p that on input (x,y) reports true and 198]. “At first he [Hilbert] was only angry and frus- diverges on all other inputs. Then test p with PHSR. trated, but then he began to try to deal constructively The above argument is a reduction of the halting prob- with the problem... Gödel himself felt—and ex- lem to PHS recognition, and in the same manner, harder pressed the thought in his paper—that his work did problems such as halting on all inputs can also be reduced, not contradict Hilbert’s formalistic point of view” implying that PHS recognition is not only undecidable, (Reid p. 199) but higher in the Arithmetical hierarchy, specifically Π0 2 • 1931: Gödel publishes “On Formally Undecidable -complete. Propositions of Principia Mathematica and Related Systems I”, (reprinted in Davis, 1965, p. 5ff)

2.9.3 Generalized to oracle machines • 19 April 1935: Alonzo Church publishes “An Un- solvable Problem of Elementary Number Theory”, See also: Turing jump wherein he identifies what it means for a function to be effectively calculable. Such a function will have A machine with an oracle for the halting problem can de- an algorithm, and "...the fact that the algorithm has termine whether particular Turing machines will halt on terminated becomes effectively known ...” (Davis, particular inputs, but they cannot determine, in general, 1965, p. 100) if machines equivalent to themselves will halt. • 1936: Church publishes the first proof that the More generally, there is no oracle machines with oracle Entscheidungsproblem is unsolvable. (A Note on the to some problem that can determine in general whether Entscheidungsproblem, reprinted in Davis, 1965, p. a machine with an oracle to the same problem will halt. 110.) Thus, for any oracle O, the halting problem for oracle Turing machines with an oracle to O is not O-computable. • 7 October 1936: Emil Post's paper “Finite Combi- natory Processes. Formulation I” is received. Post adds to his “process” an instruction "(C) Stop”. He 2.10 History called such a process “type 1 ... if the process it determines terminates for each specific problem.” (Davis, 1965, p. 289ff) Further information: History of algorithms • 1937: Alan Turing's paper On Computable Numbers With an Application to the Entscheidungsproblem • 1900: David Hilbert poses his “23 questions” reaches print in January 1937 (reprinted in Davis, (now known as Hilbert’s problems) at the Second 1965, p. 115). Turing’s proof departs from calcula- International Congress of Mathematicians in Paris. tion by recursive functions and introduces the notion 2.13. NOTES 9

of computation by machine. Stephen Kleene (1952) • Generic-case complexity refers to this as one of the “first examples of decision • problems proved unsolvable”. Geoffrey K. Pullum • Gödel’s incompleteness theorem • 1939: J. Barkley Rosser observes the essential equivalence of “effective method” defined by Gödel, • Kolmogorov complexity Church, and Turing (Rosser in Davis, 1965, p. 273, “Informal Exposition of Proofs of Gödel’s Theorem • P versus NP problem and Church’s Theorem”) • Termination analysis • 1943: In a paper, Stephen Kleene states that “In set- • Worst-case execution time ting up a complete algorithmic theory, what we do is describe a procedure ... which procedure neces- sarily terminates and in such manner that from the outcome we can read a definite answer, 'Yes’ or 'No,' 2.13 Notes to the question, 'Is the predicate value true?'.” [1] In none of his work did Turing use the word “halting” or • 1952: Kleene (1952) Chapter XIII (“Computable “termination”. Turing’s biographer Hodges does not have Functions”) includes a discussion of the unsolvabil- the word “halting” or words “halting problem” in his index. ity of the halting problem for Turing machines and The earliest known use of the words “halting problem” is in a proof by Davis (1958, p. 70–71): reformulates it in terms of machines that “eventu- ally stop”, i.e. halt: "... there is no algorithm for “Theorem 2.2 There exists a Turing machine deciding whether any given machine, when started whose halting problem is recursively unsolv- from any given situation, eventually stops.” (Kleene able. (1952) p. 382) “A related problem is the printing problem for a simple Turing machine Z with respect to a • 1952: "Martin Davis thinks it likely that he first symbol Sᵢ". used the term 'halting problem' in a series of lec- Davis adds no attribution for his proof, so one infers that tures that he gave at the Control Systems Labora- it is original with him. But Davis has pointed out that a tory at the University of Illinois in 1952 (letter from statement of the proof exists informally in Kleene (1952, Davis to Copeland, 12 December 2001).” (Footnote p. 382). Copeland (2004, p 40) states that: 61 in Copeland (2004) pp. 40ff) “The halting problem was so named (and it appears, first stated) by Martin Davis [cf Copeland footnote 61]... (It is often said that 2.11 Avoiding the halting problem Turing stated and proved the halting theorem in 'On Computable Numbers’, but strictly this In many practical situations, programmers try to avoid is not true).” infinite loops—they want every subroutine to finish (halt). [2] Moore, Cristopher; Mertens, Stephan (2011), The Nature In particular, in hard real-time computing, programmers of Computation, Oxford University Press, pp. 236–237, attempt to write subroutines that are not only guaranteed ISBN 9780191620805. to finish (halt), but are guaranteed to finish before the given deadline. [3] Stated without proof in: "Course notes for Data Compres- sion - Kolmogorov complexity", 2005, P.B. Miltersen, p.7 Sometimes these programmers use some general-purpose (Turing-complete) programming language, but attempt [4] Börger, Egon. “Computability, Complexity, Logic”. to write in a restricted style—such as MISRA C—that North-Holland, 1989. p. 121 makes it easy to prove that the resulting subroutines fin- ish before the given deadline. Other times these programmers apply the rule of least 2.14 References power—they deliberately use a computer language that • is not quite fully Turing-complete, often a language that Alan Turing, On computable numbers, with an ap- guarantees that all subroutines are guaranteed to finish, plication to the Entscheidungsproblem, Proceedings such as Coq. of the London Mathematical Society, Series 2, Vol- ume 42 (1937), pp 230–265, doi:10.1112/plms/s2- 42.1.230. — Alan Turing, On Computable Num- bers, with an Application to the Entscheidungsprob- 2.12 See also lem. A Correction, Proceedings of the London Mathematical Society, Series 2, Volume 43 (1938), • Busy beaver pp 544–546, doi:10.1112/plms/s2-43.6.544 . Free 10 CHAPTER 2. HALTING PROBLEM

online version of both parts This is the epochal pa- • John Hopcroft and Jeffrey Ullman, Introduction per where Turing defines Turing machines, formu- to Automata Theory, Languages and Computa- lates the halting problem, and shows that it (as well tion, Addison-Wesley, Reading Mass, 1979. See as the Entscheidungsproblem) is unsolvable. Chapter 7 “Turing Machines.” A book centered around the machine- of “languages”, • Sipser, Michael (2006). “Section 4.2: The Halting NP-Completeness, etc. Problem”. Introduction to the Theory of Computa- tion (Second ed.). PWS Publishing. pp. 173–182. • Andrew Hodges, Alan Turing: The Enigma, Simon ISBN 0-534-94728-X. and Schuster, New York. Cf Chapter “The Spirit of Truth” for a history leading to, and a discussion of, • c2:HaltingProblem his proof. • B. Jack Copeland ed. (2004), The Essential Turing: • Constance Reid, Hilbert, Copernicus: Springer- Seminal Writings in Computing, Logic, Philosophy, Verlag, New York, 1996 (first published 1970). Fas- Artificial Intelligence, and Artificial Life plus The Se- cinating history of German mathematics and physics crets of Enigma, Clarendon Press (Oxford Univer- from 1880s through 1930s. Hundreds of names fa- sity Press), Oxford UK, ISBN 0-19-825079-7. miliar to mathematicians, physicists and engineers appear in its pages. Perhaps marred by no overt ref- • Davis, Martin (1965). The Undecidable, Basic Pa- erences and few footnotes: Reid states her sources pers on Undecidable Propositions, Unsolvable Prob- were numerous interviews with those who person- lems And Computable Functions. New York: Raven ally knew Hilbert, and Hilbert’s letters and papers. Press.. Turing’s paper is #3 in this volume. Papers include those by Godel, Church, Rosser, Kleene, • Edward Beltrami, What is Random? Chance and or- and Post. der in mathematics and life, Copernicus: Springer- Verlag, New York, 1999. Nice, gentle read for the • Davis, Martin (1958). Computability and Unsolv- mathematically inclined non-specialist, puts tougher ability. New York: McGraw-Hill.. stuff at the end. Has a Turing-machine model in it. • Alfred North Whitehead and Bertrand Russell, Prin- Discusses the Chaitin contributions. cipia Mathematica to *56, Cambridge at the Univer- • Ernest Nagel and James R. Newman, Godel’s Proof, sity Press, 1962. Re: the problem of paradoxes, the New York University Press, 1958. Wonderful writ- authors discuss the problem of a set not be an object ing about a very difficult subject. For the mathemat- in any of its “determining functions”, in particular ically inclined non-specialist. Discusses Gentzen's “Introduction, Chap. 1 p. 24 "...difficulties which proof on pages 96–97 and footnotes. Appendices arise in formal logic”, and Chap. 2.I. “The Vicious- discuss the Peano Axioms briefly, gently introduce Circle Principle” p. 37ff, and Chap. 2.VIII. “The readers to formal logic. Contradictions” p. 60ff. • Taylor Booth, Sequential Machines and Automata • Martin Davis, “What is a computation”, in Math- Theory, Wiley, New York, 1967. Cf Chapter 9, ematics Today, Lynn Arthur Steen, Vintage Books Turing Machines. Difficult book, meant for elec- (Random House), 1980. A wonderful little paper, trical engineers and technical specialists. Discusses perhaps the best ever written about Turing Machines recursion, partial-recursion with reference to Turing for the non-specialist. Davis reduces the Turing Ma- Machines, halting problem. Has a Turing Machine chine to a far-simpler model based on Post’s model model in it. References at end of Chapter 9 catch of a computation. Discusses Chaitin proof. Includes most of the older books (i.e. 1952 until 1967 in- little biographies of Emil Post, Julia Robinson. cluding authors Martin Davis, F. C. Hennie, H. Her- mes, S. C. Kleene, M. Minsky, T. Rado) and various • Marvin Minsky, Computation, Finite and Infinite technical papers. See note under Busy-Beaver Pro- Machines, Prentice-Hall, Inc., N.J., 1967. See chap- grams. ter 8, Section 8.2 “The Unsolvability of the Halting Problem.” Excellent, i.e. readable, sometimes fun. • Busy Beaver Programs are described in Scientific A classic. American, August 1984, also March 1985 p. 23. A reference in Booth attributes them to Rado, • Roger Penrose, The Emperor’s New Mind: Concern- T.(1962), On non-computable functions, Bell Sys- ing computers, Minds and the Laws of Physics, Ox- tems Tech. J. 41. Booth also defines Rado’s Busy ford University Press, Oxford England, 1990 (with Beaver Problem in problems 3, 4, 5, 6 of Chapter 9, corrections). Cf: Chapter 2, “Algorithms and Tur- p. 396. ing Machines”. An over-complicated presentation (see Davis’s paper for a better model), but a thor- • David Bolter, Turing’s Man: Western Culture in the ough presentation of Turing machines and the halt- Computer Age, The University of North Carolina ing problem, and Church’s Lambda Calculus. Press, Chapel Hill, 1984. For the general reader. 2.15. EXTERNAL LINKS 11

May be dated. Has yet another (very simple) Turing Machine model in it. • Stephen Kleene, Introduction to Metamathematics, North-Holland, 1952. Chapter XIII (“Computable Functions”) includes a discussion of the unsolvabil- ity of the halting problem for Turing machines. In a departure from Turing’s terminology of circle-free nonhalting machines, Kleene refers instead to ma- chines that “stop”, i.e. halt.

• Logical Limitations to Machine Ethics, with Con- sequences to Lethal Autonomous Weapons - paper discussed in: Does the Halting Problem Mean No Moral Robots?

2.15 External links

• Scooping the loop snooper - a poetic proof of unde- cidability of the halting problem • animated movie - an animation explaining the proof of the undecidability of the halting problem • A 2-Minute Proof of the 2nd-Most Important The- orem of the 2nd Millennium - a proof in only 13 lines 12 CHAPTER 2. HALTING PROBLEM

2.16 Text and image sources, contributors, and licenses

2.16.1 Text

• Post correspondence problem Source: https://en.wikipedia.org/wiki/Post_correspondence_problem?oldid=729555309 Contributors: Jan Hidders, Arvindn, Ixfd64, Rodney Topor, Dcoetzee, Ancheta Wis, Giftlite, Gro-Tsen, Mboverload, Nayuki, Neilc, Gachet, Ascánder, Rjmccall, Caesura, Ruud Koot, MushroomCloud, Thatha, Marudubshinki, Rjwilmsi, MarSch, NekoDaemon, Gparker, YurikBot, Trova- tore, R.e.s., Misza13, PhS, That Guy, From That Show!, SmackBot, BiT, Chris the speller, DMacks, James Van Boxtel, Geh, Chrisahn, Cydebot, Xprotocol, Headbomb, JAnDbot, Jestbiker, Pendragon81, TXiKiBoT, Aaron Rotenberg, SieBot, Structural Induction, Sambrow, Addbot, DOI bot, IOLJeff, Yobot, Citation bot, Citation bot 1, Trappist the monk, RjwilmsiBot, Alph Bot, Twistedcerebrum, Rename- dUser01302013, ZéroBot, Frietjes, BG19bot, ChrisGualtieri, Deltahedron, Jochen Burghardt, Igor potapov, PIerre.Lescanne, Fschwarze, Monkbot, GSS-1987 and Anonymous: 24 • Halting problem Source: https://en.wikipedia.org/wiki/Halting_problem?oldid=724849022 Contributors: Damian Yerrick, AxelBoldt, Derek Ross, LC~enwiki, Vicki Rosenzweig, Wesley, Robert Merkel, Jan Hidders, Andre Engels, Wahlau, Hephaestos, Stevertigo, Michael Hardy, Chris-martin, Dominus, Cole Kitchen, Wapcaplet, Graue, Fwappler, JeremyR, Cyp, Muriel Gottrop~enwiki, Salsa Shark, Qed, Rotem Dan, Ehn, Charles Matthews, Timwi, Dcoetzee, Dysprosia, Doradus, Furrykef, David.Monniaux, Phil Boswell, DaleNixon, Robbot, Craig Stuntz, RedWolf, Cogibyte, MathMartin, Rursus, Paul G, Guillermo3, Tea2min, Ramir, Solver, Ancheta Wis, Giftlite, DavidCary, Dratman, Siroxo, Rchandra, Neilc, DNewhall, Sam Hocevar, Gazpacho, PhotoBox, Mormegil, Rich Farmbrough, Avriette, Guanabot, ArnoldReinhold, Roodog2k, DcoetzeeBot~enwiki, Bender235, Pt, Jantangring, Army1987, Cyclist, Enric Naval, Obradovic Goran, Jum- buck, Hackwrench, Axl, Sligocki, Suruena, Oleg Alexandrov, Ataru, MattGiuca, Ruud Koot, GregorB, Ryansking, Wulfila, Tslocum, Graham87, BD2412, Zoz, Oddcowboy, Koavf, Bubba73, SLi, FlaBot, Mathbot, NekoDaemon, Jameshfisher, Chobot, Adam Lindberg, Banaticus, YurikBot, Wavelength, Hairy Dude, RussBot, Spl, Robert A West, Trovatore, R.e.s., ZacBowling, Eighty~enwiki, Thiseye, Ths- grn, Muu-karhu, Hv, Zwobot, Bota47, DaveWF, Arthur Rubin, Abeliano, Claygate, Tsiaojian lee, True Pagan Warrior, SmackBot, Radak, Stux, InverseHypercube, Alksub, Eskimbot, Canderra, Ohnoitsjamie, Hmains, SpaceDude, Thumperward, Jfsamper, Torzsmokus, Lion- tooth, Jgoulden, Anatoly Vorobey, Byelf2007, Lambiam, Wvbailey, Nagle, BenRayfield, Meor, Dan Gluck, WAREL, Zero sharp, Atreys, FatalError, CRGreathouse, CBM, Chrisahn, Myasuda, Gregbard, Stormwyrm, Cydebot, Mon4, UberScienceNerd, Jdvelasc, Thijs!bot, Amlz, VictorAnyakin, JAnDbot, LeedsKing, PhilipHunt, .anacondabot, Yurei-eggtart, Singularity, MetsBot, David Eppstein, Ekotkie, Projectstann, R'n'B, Maurice Carbonaro, Aqwis, Tatrgel, Bah23, Billinghurst, Mikez302, Sapphic, HiDrNick, Macaw3, SieBot, Ncapito, Likebox, Yahastu, TimMorley, Svick, SuperMarioBrah, CBM2, ClueBot, Jeffreykegler, James Lednik, Catuila, AmirOnWiki, Gunder- sen53, XLinkBot, Luke.mccrohon, SilvonenBot, Addbot, Ijriims, Ronhjones, Fieldday-sunday, Download, Verbal, PV=nRT, Yobot, Pcap, PMLawrence, Bility, AnomieBOT, Garga2, PiracyFundsTerrorism, Mahtab mk, LilHelpa, Xqbot, Martnym, Nippashish, Ehird, Thehelp- fulbot, FrescoBot, Arlen22, Kwiki, Pinethicket, Rushbugled13, The.megapode, RedBot, Joshtch, Full-date unlinking bot, Proof Theorist, Specs112, EmausBot, John of Reading, BillyPreset, Quondum, Petersuber, Zustra, Erget2005, ChuispastonBot, Jiri 1984, Widr, Helpful Pixie Bot, BG19bot, Bengski68, Pedro.atmc, Marler8997, S.Chepurin, BattyBot, Farhanarrafi, IjonTichyIjonTichy, Enterprisey, Jochen Burghardt, Blackbombchu, KasparBot, Risc64 and Anonymous: 202

2.16.2 Images

2.16.3 Content license

• Creative Commons Attribution-Share Alike 3.0