Decidability: Church-Turing Thesis • While there are a countably infinite number of languages that are described by TMs over some alphabet Σ∗, there are an uncountably infinite number that are not • Are there machines that are more powerful than TMs that can describe some of these other languages? { Note: Since there can only be a countably infinite number of machines that are described by some finite representation, there will always be some languages that cannot be described • Theory of computation based on the concept of algorithms • Important concepts relating to theory of computation introduced in the early 20th century 1. Is it possible to axiomatize all mathematical structures of interest so that every true statement becomes a theorem? { The set of axioms may be infinite, but they must be decidable - there must be an algorithm that can determine whether a string is an axiom 2. Given a set of axioms, does there exist a machine that can decide whether an axiom is a theorem? • These questions were proved to be unanswerable 1. Godel's Incompleteness Theorem { Statement: Given a potentially infinite set of true statements in number theory (Peano arithmetic), some cannot be proven 2. Entscheidungsproblem (decision problem) of Hilbert and Achermann { Statement: (a) Does there exist an algorithm for deciding, for an arbitrary sentence w in first order logic, whether w is valid? (b) Given a set of axioms A and a sentence w, does there exist an algo- rithm for deciding whether w is entailed by A? (c) Given a set of axioms A and a sentence w, does there exist an algo- rithm for deciding whether w can be proved from A? 1 Decidability: Church-Turing Thesis (2) • Concept of algorithm is important for proof of the Entscheidungsproblem { Turing and Church independently formalized what is meant by algorithm 1. Turing's approach was the Turing machine 2. Church's approach was the lambda calculus • Church-Turing Thesis { Statement: Lambda calculus and Turing machines are equivalent in power { Generalization: Any formalisms capable of describing computational algo- rithms are equivalent in power { This has not been proven, hence it is a thesis, not a theorem { Implications are that 1. There is no computational model that is more powerful 2. There are problems (languages) that have no computational solutions { Thus, there are limitations to what computers can do 2 Decidability: The Halting Problem • The theory of computation is concerned with the classes D, SD, SD=D = SD − D, and :SD • The halting problem is concerned with the language H = f< M; w >: TM M halts on input string wg • Our concern here - and throughout the rest of this section - is whether a language is 2 D; 2 SD=D; or 2 :SD? • While correct syntax of strings input to decision procedures (i.e., < M; w >) is important, that is not our concern here { Our concern is whether machine M halts when it runs on input w • We also do not consider general strings from Σ∗ { Thus, :H = f< M; w >: TM M does not halt on wg • The halting problem is: { Is H decidable, or semidecidable? • Theorem 19.1 { Statement: Language H = f< M; w >: TM M halts on input string wg 2 SD { Proof: By construction Boolean MSH (<M, w>) f M(w); return TRUE; g 3 Decidability: The Halting Problem (2) • Theorem 19.2 { Statement: Language H = f< M; w >: TM M halts on input string wg 2= D { Proof: ∗ If H were decidable, there would be a TM MH that decides H ∗ It could be implemented as Boolean MH (string <M>, string <w>) f if (<M> represents a TM that halts on w) return TRUE; else return FALSE; g ∗ Now consider TM C: Boolean C (<M>) f if (MH (<M>, <M>)) loop; else halt; g ∗ The crux of the proof is the answer to the question · What is the result of C(< C >)? 1. If C were decidable, C would never halt 2. If C were not decidable, C would halt ∗ The premise on which C is based - the existence of MH - must be incor- rect ∗ Therefore, MH does not exist 4 Decidability: The Halting Problem (3) { Another way of looking at things: ∗ Consider the table below, which indicates encodings which a TM accepts < M1 > < M2 > < M3 > < M4 > ... M1 accept accept ... M2 accept ... M3 accept accept accept ... M4 accept ... ... ... ∗ Now consider this table, which indicates the result of running MH on each (row; col) pair < M1 > < M2 > < M3 > < M4 > ... M1 accept accept ... M2 reject reject accept ... M3 accept accept accept ... M4 accept reject ... ... ... ∗ Since C is a TM, it must appear in some row, and its encoding in some column ∗ The values in C's row are opposite the values of the diagonal 1. If [1; 1] is accept 2. Then [c; 1] is reject < M1 > < M2 > < M3 > < M4 > ... M1 accept accept ... M2 reject reject accept ... M3 accept accept accept ... M4 accept reject ... ... ... C reject accept reject accept ... ... ... ∗ What value should be stored in [c; c]? · If [c; c] is reject, it should be accept, and vice-versa! 5 Decidability: The Halting Problem (4) • Theorem 19.3 { Statement: If H were in D, then every language in SD would be in D too { Discussion: ∗ Let L 2 SD ∗ Then there exists a TM ML that semidecides L ∗ Suppose H were in D ∗ Then it would be decidable by some machine O ∗ To decide whether some w 2 L, we could use O to determine whether ML halts on w ∗ If the answer were yes, then we could run ML on w to see if it accepts { Proof: By construction of M 0 to decide above ∗ Construction Boolean M' (string w) f result = O(<ML, w>); if (result == accept) f result2 = ML(w); if (result2 == accept) accept(); else reject(); g else reject(); g ∗ O always halts, so M 0 always halts ∗ Therefore, if H were in D, all languages in SD would be too 6 Decidability: The Relationship Between D and SD • The following theorems explore the relations among the languages D, SD, and :SD • Theorem 20.1 { Statement: The Context Free Grammars are a proper subset of D { Proof: ∗ By Theorem 14.1, there is a decision procedure to determine whether a string w 2 L, where L is a CFL ∗ But AnBnCn is not a CFL, and it is decidable ∗ Therefore CF Ls ⊂ D • Theorem 20.2 { Statement: Every language L 2 D is also in SD { Proof: ∗ If L 2 D, L is decided by some TM M ∗ This means that M accepts all strings in L and only strings in L ∗ This meets the definition of semidecidability, so L 2 SD 7 Decidability: The Relationship Between D and SD (2) • Theorem 20.3 { Statement: There are languages that are not in SD { Lemma: There are a countably infinite number of semidecidable languages over an alphabet Σ { Proof (of lemma): ∗ Every language L 2 SD is semidecided by some TM with alphabet Σ ∗ The TMs with alphabet Σ can be lexicographically ordered ∗ By Theorem A.1, there are a countably infinite number of semideciding TMs ∗ Therefore, there are at most a countably infinite number of languages in SD, since there cannot be more semidecidable languages than TMs ∗ There is not a 1:1 correspondence between TMs and semidecidable lan- guages, since there is more than 1 TM to semidecide a language ∗ The number of semidecidable languages is at least countably infinite because D ⊂ SD, and D is countably infinite ∗ Therefore, there are at least and at most a countably infinite number of semidecidable languages { Proof (of theorem): ∗ By Theorem 2.2, there are an uncountably infinite number of languages over an alphabet. ∗ So there must be more languages over an alphabet than there are in SD ∗ So there is at least 1 language in :SD • Theorem 20.4 { Statement: D is closed under complement { Proof: By construction ∗ Let L 2 D ∗ There must be a deterministic TM that decides L ∗ Construct M 0 as follows 1. Initially, let M 0 = M 2. Make the non-accepting states accepting 3. Make the accepting states non-accepting ∗ M 0 now rejects what M accepts, and vice-versa ∗ Since M always halts, M 0 always halts: M 0 decides :L 8 Decidability: The Relationship Between D and SD (3) • Theorem 20.5 { Statement: SD is not closed under complement { Proof: By contradiction ∗ Suppose SD were closed under complementation ∗ Then :L 2 SD would mean that there exists a TM M 0 that semidecides :L ∗ Construct M 00 as follows 1. M 00 simulates M and M 0 in parallel on input w 2. Since w 2 L or w 2 :L, w is accepted by either M or M 0 ∗ This means that M 00 will halt on w ∗ So if all L 2 SD were closed under complement, all would also be decidable ∗ But this is not the case, so SD is not closed under complement • Theorem 20.6 { Statement: Language L 2 D iff L 2 SD and :L 2 SD { Proof(if part): ∗ If L 2 D, then L 2 SD ∗ By Theorem 20.4, :L 2 D ∗ Therefore, :L 2 SD { Proof(only-if part): By construction ∗ Since L 2 SD and :L 2 SD, both are semidecided by some TM ∗ Call these TMs M and M 0 ∗ Construct M 00 as follows 1. M 00 simulates M and M 0 in parallel on input w ∗ Since w is accepted by one of M and M 0, M 00 must halt ∗ Since M 00 decides L, L 2 D • Theorem 20.7 { Statement: :H2 = SD { Proof: ∗ H 2 SD by Theorem 19.1 ∗ If :H 2 SD, then H 2 D by Theorem 20.6 ∗ But H2 = D by Theorem 19.2 ∗ Therefore, :H2 = SD 9 Decidability: And Enumeration • TM M enumerates a language L if 1.