Decidability: Church-Turing Thesis • While There Are a Countably Infinite

Decidability: Church-Turing Thesis • While there are a countably infinite number of languages that are described by TMs over some alphabet Σ∗, there are an uncountably infinite number that are not • Are there machines that are more powerful than TMs that can describe some of these other languages? – Note: Since there can only be a countably infinite number of machines that are described by some finite representation, there will always be some languages that cannot be described • Theory of computation based on the concept of algorithms • Important concepts relating to theory of computation introduced in the early 20th century 1. Is it possible to axiomatize all mathematical structures of interest so that every true statement becomes a theorem? – The set of axioms may be infinite, but they must be decidable - there must be an algorithm that can determine whether a string is an axiom 2. Given a set of axioms, does there exist a machine that can decide whether an axiom is a theorem? • These questions were proved to be unanswerable 1. Godel’s Incompleteness Theorem – Statement: Given a potentially infinite set of true statements in number theory (Peano arithmetic), some cannot be proven 2. Entscheidungsproblem (decision problem) of Hilbert and Achermann – Statement: (a) Does there exist an algorithm for deciding, for an arbitrary sentence w in first order logic, whether w is valid? (b) Given a set of axioms A and a sentence w, does there exist an algorithm for deciding whether w is entailed by A? (c) Given a set of axioms A and a sentence w, does there exist an algorithm for deciding whether w can be proved from A?

1 Decidability: Church-Turing Thesis (2)

• Concept of algorithm is important for proof of the Entscheidungsproblem – Turing and Church independently formalized what is meant by algorithm 1. Turing’s approach was the Turing machine 2. Church’s approach was the lambda calculus • Church-Turing Thesis – Statement: Lambda calculus and Turing machines are equivalent in power – Generalization: Any formalisms capable of describing computational algorithms are equivalent in power – This has not been proven, hence it is a thesis, not a theorem – Implications are that 1. There is no computational model that is more powerful 2. There are problems (languages) that have no computational solutions – Thus, there are limitations to what computers can do

2 Decidability: The Halting Problem • The theory of computation is concerned with the classes D, SD, SD/D = SD − D, and ¬SD

• The halting problem is concerned with the language H = {< M, w >: TM M halts on input string w} • Our concern here - and throughout the rest of this section - is whether a language is ∈ D, ∈ SD/D, or ∈ ¬SD? • While correct syntax of strings input to decision procedures (i.e., < M, w >) is important, that is not our concern here – Our concern is whether machine M halts when it runs on input w • We also do not consider general strings from Σ∗ – Thus, ¬H = {< M, w >: TM M does not halt on w} • The halting problem is: – Is H decidable, or semidecidable? • Theorem 19.1 – Statement: Language H = {< M, w >: TM M halts on input string w} ∈ SD – Proof: By construction

Boolean MSH () { M(w); return TRUE; }

3 Decidability: The Halting Problem (2)

• Theorem 19.2 – Statement: Language H = {< M, w >: TM M halts on input string w} ∈/ D – Proof:

∗ If H were decidable, there would be a TM MH that decides H ∗ It could be implemented as

Boolean MH (string , string ) { if ( represents a TM that halts on w) return TRUE; else return FALSE; }

∗ Now consider TM C: Boolean C () { if (MH (, )) loop; else halt; }

∗ The crux of the proof is the answer to the question · What is the result of C(< C >)? 1. If C were decidable, C would never halt 2. If C were not decidable, C would halt

∗ The premise on which C is based - the existence of MH - must be incor- rect

∗ Therefore, MH does not exist

4 Decidability: The Halting Problem (3) – Another way of looking at things: ∗ Consider the table below, which indicates encodings which a TM accepts

< M1 > < M2 > < M3 > < M4 > ... M1 accept accept ... M2 accept ... M3 accept accept accept ... M4 accept ......

∗ Now consider this table, which indicates the result of running MH on each (row, col) pair

< M1 > < M2 > < M3 > < M4 > ... M1 accept accept ... M2 reject reject accept ... M3 accept accept accept ... M4 accept reject ......

∗ Since C is a TM, it must appear in some row, and its encoding in some column ∗ The values in C’s row are opposite the values of the diagonal 1. If [1, 1] is accept 2. Then [c, 1] is reject

< M1 > < M2 > < M3 > < M4 > ... M1 accept accept ... M2 reject reject accept ... M3 accept accept accept ... M4 accept reject ...... C reject accept reject accept ......

∗ What value should be stored in [c, c]? · If [c, c] is reject, it should be accept, and vice-versa!

5 Decidability: The Halting Problem (4)

• Theorem 19.3 – Statement: If H were in D, then every language in SD would be in D too – Discussion: ∗ Let L ∈ SD

∗ Then there exists a TM ML that semidecides L ∗ Suppose H were in D ∗ Then it would be decidable by some machine O ∗ To decide whether some w ∈ L, we could use O to determine whether ML halts on w ∗ If the answer were yes, then we could run ML on w to see if it accepts – Proof: By construction of M 0 to decide above ∗ Construction Boolean M’ (string w) { result = O(); if (result == accept) { result2 = ML(w); if (result2 == accept) accept(); else reject(); } else reject(); }

∗ O always halts, so M 0 always halts ∗ Therefore, if H were in D, all languages in SD would be too

6 Decidability: The Relationship Between D and SD • The following theorems explore the relations among the languages D, SD, and ¬SD • Theorem 20.1 – Statement: The Context Free Grammars are a proper subset of D – Proof: ∗ By Theorem 14.1, there is a decision procedure to determine whether a string w ∈ L, where L is a CFL ∗ But AnBnCn is not a CFL, and it is decidable ∗ Therefore CF Ls ⊂ D • Theorem 20.2 – Statement: Every language L ∈ D is also in SD – Proof: ∗ If L ∈ D, L is decided by some TM M ∗ This means that M accepts all strings in L and only strings in L ∗ This meets the deﬁnition of semidecidability, so L ∈ SD

7 Decidability: The Relationship Between D and SD (2)

• Theorem 20.3 – Statement: There are languages that are not in SD – Lemma: There are a countably infinite number of semidecidable languages over an alphabet Σ – Proof (of lemma): ∗ Every language L ∈ SD is semidecided by some TM with alphabet Σ ∗ The TMs with alphabet Σ can be lexicographically ordered ∗ By Theorem A.1, there are a countably infinite number of semideciding TMs ∗ Therefore, there are at most a countably infinite number of languages in SD, since there cannot be more semidecidable languages than TMs ∗ There is not a 1:1 correspondence between TMs and semidecidable languages, since there is more than 1 TM to semidecide a language ∗ The number of semidecidable languages is at least countably infinite because D ⊂ SD, and D is countably infinite ∗ Therefore, there are at least and at most a countably infinite number of semidecidable languages – Proof (of theorem): ∗ By Theorem 2.2, there are an uncountably infinite number of languages over an alphabet. ∗ So there must be more languages over an alphabet than there are in SD ∗ So there is at least 1 language in ¬SD • Theorem 20.4 – Statement: D is closed under complement – Proof: By construction ∗ Let L ∈ D ∗ There must be a deterministic TM that decides L ∗ Construct M 0 as follows 1. Initially, let M 0 = M 2. Make the non-accepting states accepting 3. Make the accepting states non-accepting ∗ M 0 now rejects what M accepts, and vice-versa ∗ Since M always halts, M 0 always halts: M 0 decides ¬L

8 Decidability: The Relationship Between D and SD (3) • Theorem 20.5 – Statement: SD is not closed under complement – Proof: By contradiction ∗ Suppose SD were closed under complementation ∗ Then ¬L ∈ SD would mean that there exists a TM M 0 that semidecides ¬L ∗ Construct M 00 as follows 1. M 00 simulates M and M 0 in parallel on input w 2. Since w ∈ L or w ∈ ¬L, w is accepted by either M or M 0 ∗ This means that M 00 will halt on w ∗ So if all L ∈ SD were closed under complement, all would also be decidable ∗ But this is not the case, so SD is not closed under complement • Theorem 20.6 – Statement: Language L ∈ D iﬀ L ∈ SD and ¬L ∈ SD – Proof(if part): ∗ If L ∈ D, then L ∈ SD ∗ By Theorem 20.4, ¬L ∈ D ∗ Therefore, ¬L ∈ SD – Proof(only-if part): By construction ∗ Since L ∈ SD and ¬L ∈ SD, both are semidecided by some TM ∗ Call these TMs M and M 0 ∗ Construct M 00 as follows 1. M 00 simulates M and M 0 in parallel on input w ∗ Since w is accepted by one of M and M 0, M 00 must halt ∗ Since M 00 decides L, L ∈ D • Theorem 20.7 – Statement: ¬H/∈ SD – Proof: ∗ H ∈ SD by Theorem 19.1 ∗ If ¬H ∈ SD, then H ∈ D by Theorem 20.6 ∗ But H/∈ D by Theorem 19.2 ∗ Therefore, ¬H/∈ SD

9 Decidability: And Enumeration • TM M enumerates a language L if 1. It starts with a blank tape 2. Writes some string w ∈ L onto the tape 3. Signals that it has written a string 4. Repeats • If L is finite, M will halt • M has a special state p to indicate when a string has been generated • M enumerates L iff, for some fixed state p, ∗ L = {w :(s, 2) `M (p, w)} • Language L is said to be Turing-enumerable iff there exists a TM that enumerates it • Theorem 20.8 – Statement: A language L is semidecidable iff it is Turing-enumerable – Proof (only if): By construction ∗ If L is Turing-enumerable, then there is a TM M that enumerates it ∗ Construct M 0(w) as follows 1. M 0 copies input w onto a second tape 2. M 0 executes M 3. On each new string w0 generated by M, M 0 compares w with w0 4. If w == w0, M 0 accepts ∗ Since M 0 semidecides L, M 0 ∈ SD

10 Decidability: And Enumeration (2) – Proof (if): Discussion ∗ If L ∈ SD, then there is a TM M that semidecides it ∗ Generate all strings in Σ∗ lexicographically ∗ Run M on each string w · If M accepts, w ∈ L ∗ Problem is that M may not halt on a given input ∗ Solution is to use technique called dovetailing · M 0 controls M so that it generates strings in parallel · M will be allowed to execute one step on each string at any time · Strings will be generated in a diagonal manner

w1[0] w1[1] w2[0] w1[2] w2[1] w3[0] w1[3] w2[2] w3[1] w4[0] w1[4] w3[2] w4[1] w5[0]

∗ If M accepts a string, M 0 will output that string

∗ If M halts on a string, computation will stop on that string (w2 above) – Proof(if): By construction of M 0 as described above