Computer Science & Engineering Department IIT Kharagpur Theory of Computation: CS41001 Lecture IX
Instructor: Goutam Biswas Autumn Semester 2014-2015
0.1 Hierarchy Theorems Space, the number of tape cells, and the time, the number of computation steps are two main measure of cost of a Turing machine computation. If we increase space or time as a function of the input size, the size of the class of problems that can be solved also increases in size. It is expected that the class of languages that can be decided in linear space (space usage is linearly dependent on the input size) is larger than the class problems decidable in logspace. This type of claims are formally established in space and time hierarchy theorems. Proof of these theorems use diagonalisation techniques in a more sopheisticated way. Theorem 1. (Space Hierarchy Theorem) Let s : N0 → N0 be a space constructible function and s(n) ≥ log n. There is a language L decidable in space O(s(n)), but not in space o(s(n)). ′ ′ In other words, if s,s : N0 → N0 are space-constructible functions, where s (n)= o(s(n)), then SPACE(s′(n)) is a proper subset of SPACE(s(n)). Proof: We need a language L with two properties, (i) L is decided in space O(s(n)), but (ii) L is not decided in space o(s(n)). We use diagonalisation to establish that L is different from every language decided in o(s(n)) space. We design an O(s(n)) space bounded Turing machine D such that L(D) is different from every L(M), where M is a o(s(n)) space bounded Turing machine. The language L(D) is different from every L(M) in at least one string. The string where D differs is related to
1. If the input x ∈ {0, 1}∗ is not a description of a Turing machine, then D rejects it.
2. The function s(n) is space constructible. D computes s(n) and puts a right marker after s(n) cells, where x =
3. D simulates M on
4. If M does not halt within 2s(|
5. If the simulation of M goes beyond s(n), M is not o(s(n)) space bounded, and D may accept or reject.
There are a few important technical details that are to be taken into consideration.
1. Even if M runs in o(s(n)) space, it is an asymptotic behavior. For small values of the input length, the space usage may be more than s(n)1. In such a situation D’s space may be insufficient to complete the simulation of M on
1 For all c> 0 there is n0 ∈ N, such that the space is < cg(n) for all n ≥ n0.
1
Following is the O(s(n)) space decider D for L(D). We argue that L(D) cannot be decide by any o(s(n)) space decider. D: input x
1. if x =
2. Compute s(|x|) and mark tape space of s(|x|) cells. If there is any move of simulation beyond this mark, reject x.
3. Prepare a counter of s(|x|) bits and initialise it to 0.
4. Simulate M on x, at every step increment the counter.
5. If the counter exceeds 2s(n), reject x.
6. If M accepts x, reject x; else accept x.
Note that the set of alphabet of M may be larger than D and one cell of M may require k cells of D for simulation. So, if M runs in s′(n) space, its simulation on D will use ks′(n) space. We prove that L(D) cannot be decided in o(s(n)) space. For the sake of contradiction we assume that L(D) is decided in space s′(n)= o(s(n)) by M. Simulation of M on D runs in ′ ks (n) space. From the definition o(s(n)), we know that there is a n0 such that for all n ≥ n0, ks′(n)
′ ′ If s,s : N0 → N0 be such that s (n)= o(s(n)) and s(n) is space constructible, then
SPACE(s′(n)) ⊂ SPACE(s(n)).
So we have L ⊂ PSPACE. Also NL ⊆ SPACE((log n)2), by the Savitch’s theorem. And by the space hierarchy, SPACE((log n)2) ⊂ SPACE(n) ⊂ PSPACE. Theorem 2. (Time Hierarchy Theorem) For any time constructible function t : N0 → N0, t(n) ≥ n there is a language L that is decidable in O(t(n)) time but not decidable in o(t(n)/ log t(n)) time. [On a single tape machine]. ′ ′ In other words, if t,t : N0 → N0 are time constructible functions so that t (n) = o(t(n)/ log t(n)), then DTIME(t′(n)) ⊂ DTIME(t(n)). The idea is essentially same. We design a Turing machine D that rejects any input not of the form
2 method, other than counting, is known to keep track of the number of steps of simulation of M. Proof: D: input x 1. If x is not of the form
2. Compute t(n) and log t(n) and initialise a counter of size O(log t(n)) to t(n)/ log t(n).
3. If the counter is 0, reject.
4. Simulate next step of M on x.
5. If M accepts x, then reject; if M rejects x, then accept.
6. Decrement counter.
7. Goto step 3. QED.
We argue that step 4,5,6 can be performed in O(t(n)) time. We assume that D is a single tape machine. If we keep the step counter of size log(t(n)/ log t(n)) = O(log t(n)); information about the state, head-position, and the transition table of M (all are of constant size), far away from the head position of D, the simulation cost will increase. So after every move of M, D will carry all these required information near its head position on a separate track. It costs O(log t(n)) time. So the simulation of t(n)/ log t(n) steps of M takes O(t(n)) steps of D. The tape of D has more than one track, (i) on the first track the content of M’s tape is present, (ii) on the second track the current state and state transition function of M is stored, (iii) the step counter is kept on the third track. The content of second and third tracks are to be kept closer to the head position of D on the first track. After every move, this information is to be shifted. The length of the content of track two is fixed for a particular M. So shifting this information requires fixed number of steps at every step of simulation of M. The length of the counter in track three is O(log t(n)) and all its bits are shifted by at most one cell after every move. It requirs O(log t(n)) moves.
To prove that L(D) cannot be decided in o(t(n)/ log t(n)) time. We start with the as- sumption that there is a Turing machine M that decides L(D) in g(n)= o(t(n)/ log t(n)). D simulates M in time cg(n) steps (not considering the simulation step counter), where c is a constant. g(n) = o(t(n)/ log t(n)), so there is a n0 such that for all n>n0, cg(n) < t(n) log t(n). n So for input
Example 1. We prove a simpler statement: DTIME(n) ⊂ DTIME(n1.5). Consider the following Turing machine: D: input x 1. If x =< M10∗ >, then reject.
2. Simulates M on < M10∗ > for |x|1.4 steps of simulation.
3 3. If M does not halt on < M10∗ > within these steps, then reject.
4. If M accepts, then reject; if M rejects, then accept.
It is known that the simulation of a t(n) time bounded deterministic Turing machine M by a universal Turing machine takes c′t(n) log t(n) steps, for all input x, where |x| = n. The constant c′ depends on the alphabet size, number of states and the number of tapes of M. Even for a fixed M the length of x may be arbitraryly large. So by definition D halts within O(n1.4) steps. And L(D) ∈ DTIME(n1.5). We prove that L(D) ∈ DTIME(n). Let there be a Turing machine N and a constant c so that N decides L(D) in c|x| steps (L(D)= L(M)). 1.4 ′ There is some n0, so that n > c cn log n, for all n>n0. Even for a fixed M there are ∗ input longer than n0. So the simulation of M on x =< M10 > will terminate, as the number of simulation steps are less than n1.4. But then the decision of D will be opposite to that of M, on the input i.e. x ∈ L(D) if and only if x ∈ L(M) - a contradiction. Theorem 3. (Nondeterministic Time Hierarchy Theorem) If f, g are time constructible func- tions satisfying f(n +1) = o(g(n)), then NTIME(f(n)) ⊂ NTIME(g(n)). Following outline of proof is given is [SABB] for a special case: NTIME(n) ⊂ NTIME(n1.5).
It seems direct diagonalisation cannot be done in case of nondeterministic machines as it is not known whether nondeterministic time classes are closed under complementation. We do not know how to design a n1.5 time bounded nondeterministic Turing machine D, that will simulate a n time bounded nondeterministic Turing machine M and will “flip” accept and reject answers of M efficiently. But it is known that the complement of a NTIME(n) language can be decided in exponential time. The theorem is proved using this ’exponential time simulation’ using a method called lazy diagonalization. The machine is suppose to “flip answer” of M for one string taken from an exponentially large set of strings. f(i)1.2 Let f : N0 → N0 be a function defined as follows: f(1) = 2 and f(i + 1) = 2 . We design a nondeterministic, O(n1.5) time bounded Turing machine D that performs in the following way. Given an input of length n, a value of i is computed so that f(i) < n ≤ f(i + 1). D will “flip” the answer of the nondeterministic Turing machine Mi, machine encoded by i, on some input from the set {1n : f(i) 1. If x ∈ 1∗, reject. 2. If x = 1n, compute i such that f(i) n+1 1.1 3. If f(i) f(i)+1 1.1 4. If n = f(i + 1), simulate Mi on 1 for (f(i) + 1) steps. Accept x, if and only if f(i)+1 Mi rejects 1 within these steps. (f(i)+1)1.1 In the last step it will be necessary to visit all the 2 branches of Mi on the input 1.2 1f(i)+1. But that will not increase the running time beyond O(n1.5) as f(i + 1) = 2f(i) . So the nondeterministic Turing machine D runs for O(n1.5) time. Let L be the language decided by D. We prove that L ∈ NTIME(n). Let us assume that there is an NTM, M, 4 that decides L in O(n) time. We can find an i so that L(M) = L(Mi) (there are infinite 1.1 number of equivalent machines). Mi can be simulated within n steps, on an input of length n, f(i) f(i)+1 f(i)+1 f(i)+2 f(i+1) f(i+1) f(i)+1 Mi(1 )=A D(1 )=D Mi(1 )=A =D Mi(1 )=A D(1 ) =D Mi(1 ), where ‘=A’ is equality by our assumption of D and Mi; and ‘=D’ is equality by our design of D. This clearly is a contradiction. Large number of problems in P along with large number of NP-complete problems are known. One may think that all problems in NP are either in P or NP-complete. If P = NP, then it is trivially true. But if it is the case that P = NP, then we have the following interesting theorem. Theorem 4. (Ladner[?]) If P = NP then there exists a language L ⊆ NP \ P that is not NP-complete. 0.2 Limit of Diagonalisation We have already come across to oracle Turing machine. When we modify the original Turing machine model by supplying an oracle, we essentially assume that some information is avail- able without paying any computation cost. So it is natural that some problems are more easily solvable in this model i.e. relative to the given oracle. This defines relativised complexity classes. Definition 1: The class PL is the class of languages decided by a polynomial-time bounded oracle Turing machine with the oracle language L. The class NPL is defined similarly. Example 2. The language SAT , a complete problem of coNP belongs to PSAT . D: input x 1. If x is not a proper encoding of a CNF Boolean formula, reject. 2. Ask whether x ∈ SAT . 3. If not, accept, else reject. Finally we conclude that through Karp reduction every element of coNP belongs to PSAT . On the basis of following two propositions, people believe that the question of equality or non-equality of P and NP cannot be resolved by relativisation. Two essential facts about a Turing machine are, (i) there is an encoding of a Turing machine, and (ii) an encoding can be used to simulate the machine using a universal machine with a small added cost of simulation. For a proof of diagonalisation, a machine M simulates another machine N, and depending on the outcome of the simulation behaves differently. We replace M and N be oracle Turing machines, and supply the same oracle to both of them. Any query of the simulated machine can be made by the simulator and the simulation can 5 proceed as earlier. So any result proved by diaginalisation on ordinary Turing machine will also hold for its oracle counter part. If there is a proof of P = NP by diagonalisation of ordinary Turing machine, then the proof can be extended for the relativised classes with any oracle i.e. PB = NPB, for any B. But we have the following proposition. Proposition 5. There is a language B such that PB = NPB. Proof: Whatever be the language B, PB ⊆ NPB. We prove the other direction of the subset relation by taking B as a PSPACE-complete language2. Let L ∈ NPB. The language is clearly in PSPACE as any nondeterministic polynomial time computation along with query to a PSPACE-complete language B can be simulated in PSPACE. We know that L is Karp reducible to B by a polynomial-time function f. We have the fol- lowing polynomial time oracle decider for L. D: input x 1. Compute f(x). 2. Ask whether f(x) ∈ B. 3. If the answer is ‘yes’, accept; else reject. As f(x) can be computed in polynomial time, L ∈ PB. QED. Similarly we cannot prove that P = NP by simulation alone. As in that case, the rela- tivised classes will also be equal for any oracle. But we have the following proposition. Proposition 6. There is a language A such that PA = NPA. Proof: For any language A we define a language LA as follows: ∗ LA = {x ∈ {0, 1} : ∃y ∈ A, s.t. |x| = |y|}. ∗ All strings of length n ≥ 0 from {0, 1} is in LA if and only if there is a string of length n in A A. We claim that LA ∈ NP . N: input x 1. Nondeterministically guess a string y whose length is equal to that of x. 2. Ask the oracle about the membership of y in A. 3. If the answer is ‘yes’, accept x. A We wish to choose A in such a way that LA is not in P . Let M1, M2, be an enumeration of all polynomial time Oracle Turing machines. For simplicity we assume that the running i th time of Mi is n . The construction of A is done in stages. Before the i stage, i = 1, 2, , there are a finite number of strings of {0, 1}∗, that have been decided to be (or not to be) in A A, such that Mj ’s, j = 1, 2, , i − 1, do not decide LA. Stage i: Up to the (i − 1)th stage we have included or excluded a finite number of strings in A. So their membership status in A are known. Choose an n greater than the lengths of 2 We can also take any EXP-complete language e.g. Lexp. 6 all strings whose membership in A has already been decided, and 2n > ni, where ni is the A n running time of Mi. We extend the knowledge of A so that Mi accepts 1 if and only if n 1 ∈ LA. A n A The oracle Turing machine Mi is simulated on the input 1 . During the simulation, Mi sends queries to the oracle. The simulator process the queries as follows: If the membership status of the query string y is already known (either in A or not in A), the answer is given A consistently. If the status of y is unknown, the answer (from the simulater to Mi ) is uniformly A ‘no’ i.e. the query string does not belong to A. Simulation continues until Mi halts. The set A n n A is ‘constructed’ one step further so that Mi accepts 1 if and only if 1 ∈ LA. n By the defination 1 ∈ LA if and only if there is a string of length n in A. If the oracle A machine Mi wants to accept LA, either it must detect a string of length n in A and accept 1n, or it gets negative answers to all query for all strings of length n (2n in number). But the ni time bounded machine cannot send query for all 2n strings of length n and get answer A n ‘no’. So Mi halts and takes decision on its input 1 on insufficient information. At the end of every stage of simulation the set A is extended to ensures that the decision A n A of Mi on 1 is incorrect, so that Mi cannot accept LA. n n If 1 is accepted by Mi, then no string of length n is ‘put’ in A. This implies that 1 ∈ LA. n If 1 is rejected by Mi, then one string of length n for which Mi did not send a query, so its A n status is unknown to Mi , is put in A. It implies that 1 ∈ LA. Note that such a string must exist as there are 2n strings of length n and 2n >ni. After this the simulation proceeds to the stage i + 1. If status of some of the strings are still unknown, they are out of A. So the construction of A is the limit of this process. LA for this A cannot be decided by any polynomial time bounded oracle Turing machine with A as the oracle. QED. References [CHP] Computational Complexity by Christos H Papadimitriou, Pub. Addision-Wesley, 1994, ISBN 0-201-53082-1. [NI] N Immerman, Nondeterministic space is closed under complementation, SIAM J. Com- put. 17(5):935-938, 1988. [MS] Theory of Computation by Michael Sipser, Pub. Cengage Learning, 2007, ISBN 978-81- 315-0513-7. [SABB] Computational Complexity, A Modern Approach by Sanjeev Arora & Boaz Barak, Pub. Cambridge University Press, 2009, ISBN 978-0-521-42426-4. 7