Time Complexity [Turing] has for the first time succeeded in giving an absolute definition ofan interesting epistemological notion, i.e., one not depending on the formalism chosen. – Kurt Gödel [Turing machines have] the advantage of making the identification with effectiveness in the ordinary (not explicitly defined) sense evident immediately. – Alonzo Church Computational Complexity, by Fu Yuxi Time Complexity 1 / 80 Synopsis 1. Turing Machine 2. Universal Turing Machine 3. Speedup Theorem 4. Time Complexity Class 5. Verification Problem 6. Time Hierarchy Theorem 7. Gap Theorem Computational Complexity, by Fu Yuxi Time Complexity 2 / 80 Turing Machine Computational Complexity, by Fu Yuxi Time Complexity 3 / 80 k-Tape Turing Machine A k-tape Turing Machine M has k-tapes. I The first tape is the read-only input tape. I The other k − 1 tapes are the read-write work tapes. I the k-th tape is also used as the output tape. 0 1 0 1 1 1 1 0 1 0 ··· q8 0 1 0 1 1 1 1 0 1 0 ··· ··· Computational Complexity, by Fu Yuxi Time Complexity 4 / 80 k-Tape Turing Machine A k-tape Turing Machine is described by a tuple (Γ; Q; δ). 1. A finite set Γ of symbol, called alphabet such that Γ ⊇ f0; 1; □; Bg. 2. A finite set Q of states such that Q ⊇ fqstart; qhaltg. 3. A transition function δ : Q × Γk ! Q × Γk−1 × fL; S; Rgk. 0 1 0 1 1 1 1 0 1 0 ··· q8 0 1 0 1 1 1 1 0 1 0 ··· ··· Computational Complexity, by Fu Yuxi Time Complexity 5 / 80 Configuration and Computation A configuration of a running TM M consists of the following: I the state; I the contents of the work tapes; I the head positions. initial/start configuration, final configuration; 1-step computation 0 1 0 1 1 1 1 0 1 0 ··· q8 0 1 0 1 1 1 1 0 1 0 ··· ··· Computational Complexity, by Fu Yuxi Time Complexity 6 / 80 Problems Solvable by Turing Machines A function f : f0; 1g∗ ! f0; 1g∗ is a problem. I M computes or solves f if M(x) = f(x) for every x 2 f0; 1g∗. I “M(x) = y” stands for “M halts with y written on its output tape if its input tape is preloaded with x”. A function d : f0; 1g∗ ! f0; 1g is a decision problem. I M decides d if M computes d. A set L ⊆ f0; 1g∗ is a language. 1; if x 2 L; I M accepts L if M decides the characteristic function L(x) = 0; if x 2= L: Computational Complexity, by Fu Yuxi Time Complexity 7 / 80 A 2-Tape Turing Machine for Palindrome Problem hqs; B; □i ! hqc; B; R; Ri hqc; 0; □i ! hqc; 0; R; Ri hqc; 1; □i ! hqc; 1; R; Ri hqc; □; □i ! hql; □; L; Si hql; 0; □i ! hql; □; L; Si hql; 1; □i ! hql; □; L; Si hql; B; □i ! hqt; □; R; Li hqt; 0; 0i ! hqt; □; R; Li hqt; 1; 1i ! hqt; □; R; Li hqt; □; Bi ! hqy; 1; S; Si hqt; 0; 1i ! hqn; □; S; Li hqt; 1; 0i ! hqn; □; S; Li hqn; 0; 0i ! hqn; □; S; Li hqn; 0; 1i ! hqn; □; S; Li hqn; 1; 0i ! hqn; □; S; Li hqn; 1; 1i ! hqn; □; S; Li hqn; 0; Bi ! hqn; B; S; Ri hqn; 1; Bi ! hqn; B; S; Ri hqn; 0; □i ! hqn; 0; S; Si hqn; 1; □i ! hqn; 0; S; Si Computational Complexity, by Fu Yuxi Time Complexity 8 / 80 Time Function Suppose T : N ! N and M computes the problem f. We say that M computes f in T(n)-time if its computation on every input x requires at most T(jxj) steps. I jxj denotes the length of x. I For example j21024 − 1j = 1024. We shall assume that all time functions ≥ n. Computational Complexity, by Fu Yuxi Time Complexity 9 / 80 Design Turing Machines for the following functions: 1. s(x) = x + 1. [using a TM for s(x) we can implement counter.] x 2. u(x) = 1 . [we often attach 1x to disallow a TM to run more than x steps.] x 3. e(x) = 2 . [the machine simply outputs 10x.] 4. l(x) = log(x). [if x = 2k, the machine outputs jxj + 1; otherwise it outputs jxj.] Computational Complexity, by Fu Yuxi Time Complexity 10 / 80 Time Constructible Function Suppose T : N ! N and T(n) ≥ n. 1. T is time constructible if there is a Turing Machine that computes the function 1n 7! xT(n)y in time O(T(n)). 2. T is fully time constructible if there is a Turing Machine that upon receiving 1n stops in exactly T(n)-steps. We shall only care about the time-constructible functions. Computational Complexity, by Fu Yuxi Time Complexity 11 / 80 Hard-Wiring a Clock to a Turing Machine Let M = (Q0; Γ0; !0) be a k0-tape TM. Let T = (Q1; Γ1; !1) be a k1-tape TM that runs in T(n)-steps. We can use T as a timer to force M to terminate in no more than T(n)-steps. I The integrated (k0+k1)-tape TM consists of two parallel machines. After replicating the input, it operates as the TM specified by I 1 0 Q = Q0 × Q1 and (q; q ) = qhalt for q 2 Q0 and (q ; q) = qhalt for q 2 Q1; I halt halt Γ = Γ0 × Γ1; I ! = !0 × !1. T is said to be hard-wired to M. Computational Complexity, by Fu Yuxi Time Complexity 12 / 80 Complexity theory ought to be model independent. Variants of Turing Machines are equivalent to the k-tape Turing Machines in the sense that they can simulate each other with polynomial overhead. Computational Complexity, by Fu Yuxi Time Complexity 13 / 80 Oblivious Turing Machine A TM is oblivious if the locations of its heads at the i-th step of the execution on input x depend only on jxj and i. Computational Complexity, by Fu Yuxi Time Complexity 14 / 80 Church-Turing Thesis. Every physically realizable computing device can be simulated by a Turing Machine. Law of Nature vs Wisdom of Human. Computational Complexity, by Fu Yuxi Time Complexity 15 / 80 Universal Turing Machine Computational Complexity, by Fu Yuxi Time Complexity 16 / 80 The confusion of software, hardware and datum lies at the heart of computation theory. ∗ I Finite syntactic objects can be coded up by numbers. [(101) .] Computational Complexity, by Fu Yuxi Time Complexity 17 / 80 Turing Machine as String 1. A transition (p; a; b; c) ! (q; d; e; R; S; L) can be coded up by say 001y1010y1100y0000yy011y1111y0000y01y00y10: 2. A transition table can be coded up by a string of the form z _ z ::: z _ z: (1) 3. A binary representation of (1) is obtained by using the following map: 0 7! 01; 1 7! 10; y 7! 00; z 7! 11: Computational Complexity, by Fu Yuxi Time Complexity 18 / 80 Turing Machine as String The encodings can be made to enjoy the following property: 1. Every TM is represented by infinitely many strings in f0; 1g∗. I If σ encodes a machine, then 0iσ0j encodes the same machine. 2. Every string in f0; 1g∗ represents some TM. I Let all illegal strings code up a specific TM. I Let xMy be the binary representation of TM M. I Let Mα be the TM represented by the string α. Computational Complexity, by Fu Yuxi Time Complexity 19 / 80 Effective Enumeration of Turing Machine By fixing an effective bijection from f0; 1g∗ to N, the set of natural numbers, we obtain M0; M1;:::; Mi;:::: The functions defined by these machines are normally denoted by ϕ0; ϕ1; : : : ; ϕi;:::: Computational Complexity, by Fu Yuxi Time Complexity 20 / 80 Universal Turing Machine and Efficient Simulation Theorem. There is a universal TM U that renders true the following statements. ∗ 1. U(x; α) ' Mα(x) for all x; α 2 f0; 1g . [this is Turing’s universal machine] 2. If Mα(x) halts in T(jxj) steps, then U(x; α) halts in cT(jxj) log T(jxj) steps, where c is a polynomial of α. [c is independent of jxj.] I The version with O(T(n)) time amplification appeared in 1965. I The present version was published in 1966. 1. J. Hartmanis and R. Stearns. On the Computational Complexity of Algorithms. Transactions of AMS, 117:285-306, 1965. 2. F. Hennie and R. Stearns. Two-Tape Simulation of Multitape Turing Machines. Journal of ACM, 13:533-546, 1966. Computational Complexity, by Fu Yuxi Time Complexity 21 / 80 Proof of Hennie and Stearns A universal TM has a fixed number of work tapes. How does it deal with a source TM with an unknown number of work tapes? The solution is to use two work tapes: I The main tape simulates all the work tapes of the source TM. I The scratch tape is used to record current state, to indicate zone boundaries, and to speed up shifting. Computational Complexity, by Fu Yuxi Time Complexity 22 / 80 Proof of Hennie and Stearns From three bidirectional worktapes to one bidirectional worktape f i r s t w o r k t a p e s e c o n d w o r k t a p t h i r d w o r k t a p e L2 L1 L0 0 R0 R1 R2 Imagine that the head is fixed and the tapes are shifting in opposite direction. One may perceive that a symbol stored in the main tape of U is a tuple, say (k; r; o).