Definition: A Turing Machine is a 7 tuple
T = (Q, Σ, Γ, δδδ, q 0, q accept , q reject ), where: Q is a finite set of states Σ is the input alphabet, where ∉∉∉ Σ Γ is the tape alphabet, where ∈∈∈ Γ and Σ ⊆⊆⊆ Γ δδδ : Q ××× Γ → Q ××× Γ ××× {L,R}
q0 ∈∈∈ Q is the start state Today: Undecidability, qaccept ∈∈∈ Q is the accept state Recognition, Enumeration, qreject ∈∈∈ Q is the reject state, and qreject ≠≠≠ qaccept all that good stuff
A TM recognizes a language if it accepts all w ∈∈∈ Σ* w ∈∈∈ Σ* and only those strings in the language
A language is called recognizable or TM w ∈∈∈ L ? TM w ∈∈∈ L ? recursively enumerable, (or r.e.) if some TM recognizes it yes no yes no
accept reject accept reject or no output A TM decides a language if it accepts all strings in the language and rejects all strings L is decidable L is recognizable not in the language (recursive) (recursively enumerable) A language is called decidable (or recursive) if some TM decides it Theorem: L is decidable if both L and ¬¬¬L are recursively enumerable
Recall: Given L ⊆⊆⊆ ΣΣΣ*, define ¬¬¬L := ΣΣΣ* L There are languages over {0,1} Theorem: L is decidable that are not decidable iff both L and ¬¬¬L are recognizable Assuming the Church Turing Thesis, this means there are problems that Given: NO computing device can solve a TM M1 that recognizes A and a TM M2 that recognizes ¬¬¬A, We can prove this using a counting argument. we can build a new machine M that decides A We will show there is no onto function from the set of all Turing Machines to the set of all
Simulate M 1 (x) on one tape, M 2 (x) on another. languages over {0,1}. (But it works for any Σ) One of them must halt. If M1 halts then accept. That is, every mapping from Turing machines to If M2 halts then reject. languages must “miss” some language.
1 “There are more problems to There are languages over {0,1} solve than there are programs Languages that are not recognizable to solve them.” over {0,1}
Proof Outline: Turing Machines 1. Every recognizable language can be associated with a Turing machine 2. Every TM corresponds to a bit string 3. So there is a 1 1 mapping from the set of all recognizable languages, to {0,1}* 4. But the set of all languages has a bijection with the POWER SET of {0,1}* 5. The power set of A is always larger than A, so there must be unrecognizable languages
Let L be any set and 2L be the power set of L MORAL: Theorem: There is no onto map from L to 2 L L Proof: Assume, for a contradiction, that No map from L to the power set 2 there is an onto map f : L →→→ 2L can cover all elements in 2L
Let S = { x ∈∈∈ L | x ∉∉∉ f(x) } ∈∈∈ 2L No matter what the set L is, If f is onto, then there is a y ∈∈∈ L where f(y) = S the power set 2 L always has Suppose y ∈∈∈ S. By definition of S, y ∉∉∉ f(y) = S. strictly greater cardinality than L So y ∉∉∉ S. By definition of S, y ∈∈∈ f(y) = S. Contradiction
There are undecidable (and unrecognizable) Russell’s Paradox in Set Theory languages over {0,1} In the early 1900’s, logicians were trying to {Recognizable languages over {0,1}} define consistent foundations for mathematics. {Turing Machines} {Languages over {0,1}} Suppose X = “universe of all possible objects” “Frege’s Axiom” Let P : X →→→ {0,1}. {0,1}* {Sets of strings Then {S in X | P(S) = 1} is a set. of 0s and 1s} Define F = { S ∈∈∈ X | S ∉∉∉ S } L Suppose F ∈∈∈ F. Then by definition, F ∉∉∉ F. Set L Set of all subsets of L: 2 So F ∉∉∉ F and by definition F ∈∈∈ F. This set theory is inconsistent! There are (many) unrecognizable languages
2 Theorem: There is no onto function from the + positive integers Z+ to the real numbers in (0, 1) Let Z = {1,2,3,4 }. {0,1}* Power set of {0,1}* There is a bijection between Z+ and Z+ ××× Z+ Proof: Suppose f is such a function: 1 0.28347279 2 (1,1) (1,2) (1,3) (1,4) (1,5) 2 0.88388384 8 3 0.77635284 6 (2,1) (2,2) (2,3) (2,4) (2,5) 4 0.11111111 1 (3,1) (3,2) (3,3) (3,4) (3,5) 5 0.12345678 5 : : (4,1) (4,2) (4,3) (4,4) (4,5) 1 if [ n th digit of f(n) ] ≠≠≠ 1 [ n th digit of r ] = (5,1) (5,2) (5,3) (5,4) (5,5) 0 otherwise r is never f(n) ≠≠≠ r for all n (Here, r = 0.11101... ) output by f
A Concrete Undecidable Problem
ATM = { (M, w) | M is a TM that accepts string w }
Theorem: ATM is recognizable but NOT decidable
ATM is recognizable: Define a TM U as follows: U is a universal TM On input (M, w), U runs M on w. If M ever accepts, accept. If M ever rejects, reject. Therefore,
U accepts (M,w) ⇔ M accepts w ⇔ (M,w) ∈∈∈ ATM Therefore, U recognizes ATM
ATM = { (M,w) | M is a TM that accepts string w } OUTPUT OF H(x,y)
ATM is undecidable: (proof by contradiction) M1 M2 M3 M4 D Assume machine H decides A TM M accept accept accept reject accept Accept if M accepts w 1 H( (M,w) ) = M2 reject accept reject reject reject
Reject if M does not accept w M3 accept reject reject accept accept accept Construct a new TM D as follows: on input M, M4 accept reject reject reject run H on (M,M) and output the opposite of H. : Reject if MD accepts MD D reject reject accept accept ? D( MD ) = Accept if MD does not accept MD
3 OUTPUT OF D(M) ATM = { (M,w) | M is a TM that accepts string w }
ATM is undecidable: (constructive proof) M1 M2 M3 M4 D Assume machine H recognizes ATM M acceptreject accept accept reject accept 1 Accept if M accepts w M2 reject acceptreject reject reject reject H( (M,w) ) =
M3 accept reject acceptreject accept accept Rejects or loops otherwise
M4 accept reject reject acceptreject accept Construct a new TM DH as follows: on input M, : run H on (M,M) and output the “opposite” of H whenever you get an answer from H. D reject reject accept accept ? D(M) outputs the opposite of H(M,M)
Reject if DMH accepts DM H That is: (i.e. if H( DMH , DMH ) = Accept)
Given any machine H recognizing ATM Accept if DM rejects DM D ( DM ) = H H H H (H is a potential decider for ATM ) (i.e. if H( DMH , DMH ) = Reject) we can effectively construct an instance which loops if DMH loops on DM H does not belong to A TM (namely, ( DH, DH )) (i.e. if H( DMH , D MH ) loops) but H runs forever on the input ( DH, DH ) Note: There is no contradiction here! So H cannot decide A TM DH must run forever on DH Given any program that recognizes the We can effectively construct an instance (DH, DH) Acceptance Problem, we can efficiently which does not belong to ATM construct an input where the program hangs! but H fails to tell us that: H(DH, DH) runs forever!
The Halting Problem
HALT TM = { (M,w) | M is a TM that halts on string w }
Theorem: ATM is recognizable but NOT decidable Theorem: HALT TM is undecidable Proof: Assume (for a contradiction) there is a
TM H that decides HALT TM Corollary: ¬¬¬ATM is not recognizable! We use H to construct a TM M’ that decides ATM ¬¬¬ATM = { (M,w) | M does not accept string w } M’(M,w): run H(M,w) If H rejects then reject Proof: Suppose ¬¬¬A is recognizable. TM If H accepts, run M on w until it halts: Then ¬¬¬ATM and A TM are both recognizable But that would mean they’re decidable Accept if M accepts and Reject if M rejects
4 (M,w) The Emptiness Problem
EMPTY TM = { M | M is a TM such that L(M) = ∅∅∅} (M,w) Given a program, does it always reject? If M halts Theorem: EMPTY is undecidable M’ Does M TM H w halt on w? Proof: Assume (for a contradiction) there is a
TM E that decides EMPTY TM We’ll use it to get a decider D for A TM If M doesn’t M D(M,w) := Build a TM M’ with the behavior: halt: REJECT “M’(x) := if x ≠≠≠ w reject else sim M(w)” Run E(M’). If E accepts, reject. If rejects, accept.
The Complement of Emptiness The Emptiness Problem ¬¬¬EMPTY TM = { M | M is a TM such that L(M) ≠≠≠ ∅∅∅} Given a program, does it accept some input? Theorem: EMPTY TM is undecidable
Theorem: ¬¬¬EMPTY TM is recognizable Theorem: ¬¬¬EMPTY TM is recognizable
Proof: Corollary: EMPTY is unrecognizable M’(M) := TM For all pairs of positive integers (i, t), Proof: If ¬¬¬EMPTY TM and EMPTY TM recognizable, Let x be the ith string in lex order. that would imply EMPTY is decidable Determine if M(x) accepts within t steps TM If yes then accept .
L(M) ≠≠≠ ∅ if and only if M’ halts and accepts M
One can often show that a language L Mapping Reductions is undecidable by showing that if L is f : Σ* →→→ Σ* is a computable function if there is decidable, then so is A a Turing machine M that halts with just f(w) TM written on its tape, for every input w. A language A is mapping reducible to We reduce A to the language L TM language B, written A ≤≤≤m B, if there is a computable f : Σ* →→→ Σ*, such that for every w, A ≤ L TM w ∈∈∈ A ⇔ f(w) ∈∈∈ B
f is called a mapping reduction We showed: ATM ≤ HALT TM (or many one reduction) from A to B and ATM ≤ ¬¬¬EMPTY TM
5 Next Episode:
Undecidability and Reductions Between Problems
6