Complexity Theory Anuj Dawar Texts The main texts for the course are: Computational Complexity. Christos H. Papadimitriou. Introduction to the Theory of Computation. Michael Sipser. Anuj Dawar Complexity Theory References Other useful references include: Computers and Intractability: A guide to the theory of NP-completeness. Michael R. Garey and David S. Johnson. P, NP and NP-completeness. Oded Goldreich. Computability and Complexity from a Programming Perspective. Neil Jones. Computational Complexity - A Modern Approach. Sanjeev Arora and Boaz Barak. Anuj Dawar Complexity Theory Outline A rough lecture-by-lecture guide, with relevant sections from the text by Papadimitriou (or Sipser, where marked with an S). • Algorithms and problems. 1.11.3. • Time and space. 2.12.5, 2.7. • Time Complexity classes. 7.1, S7.2. • Nondeterminism. 2.7, 9.1, S7.3. • NP-completeness. 8.18.2, 9.2. • Graph-theoretic problems. 9.3 Anuj Dawar Complexity Theory Outline - contd. • Sets, numbers and scheduling. 9.4 • coNP. 10.110.2. • Cryptographic complexity. 12.112.2. • Space Complexity 7.1, 7.3, S8.1. • Hierarchy 7.2, S9.1. • Descriptive Complexity 5.7, 8.3. Anuj Dawar Complexity Theory Algorithms and Problems Insertion Sort runs in time O(n2), while Merge Sort is an O(n log n) algorithm. The rst half of this statement is short for: If we count the number of steps performed by the Insertion Sort algorithm on an input of size n, taking the largest such number, from among all inputs of that size, then the function of n so dened is eventually bounded by a constant multiple of n2. It makes sense to compare the two algorithms, because they seek to solve the same problem. But, what is the complexity of the sorting problem? Anuj Dawar Complexity Theory Lower and Upper Bounds What is the running time complexity of the fastest algorithm that sorts a list? By the analysis of the Merge Sort algorithm, we know that this is no worse than O(n log n). The complexity of a particular algorithm establishes an upper bound on the complexity of the problem. To establish a lower bound, we need to show that no possible algorithm, including those as yet undreamed of, can do better. In the case of sorting, we can establish a lower bound of Ω(n log n), showing that Merge Sort is asymptotically optimal. Sorting is a rare example where known upper and lower bounds match. Anuj Dawar Complexity Theory Review The complexity of an algorithm (whether measuring number of steps, or amount of memory) is usually described asymptotically: Denition For functions f : IN ! IN and g : IN ! IN, we say that: • f = O(g), if there is an n0 2 IN and a constant c such that for all n > n0, f (n) ≤ cg(n); • f = Ω(g), if there is an n0 2 IN and a constant c such that for all n > n0, f (n) ≥ cg(n). • f = θ(g) if f = O(g) and f = Ω(g). Usually, O is used for upper bounds and Ω for lower bounds. Anuj Dawar Complexity Theory Lower Bound on Sorting An algorithm A sorting a list of n distinct numbers a1;:::; an. ai < aj ? ap < aq ? ak < al ? ? . ar < as . done done done done done To work for all permutations of the input list, the tree must have at least n! leaves and therefore height at least log2(n!) = θ(n log n). Anuj Dawar Complexity Theory Travelling Salesman Given • V a set of nodes. • c : V × V ! IN a cost matrix. Find an ordering v1;:::; vn of V for which the total cost: n−1 X c(vn; v1) + c(vi ; vi+1) i=1 is the smallest possible. Anuj Dawar Complexity Theory Complexity of TSP Obvious algorithm: Try all possible orderings of V and nd the one with lowest cost. The worst case running time is θ(n!). Lower bound: An analysis like that for sorting shows a lower bound of Ω(n log n). Upper bound: The currently fastest known algorithm has a running time of O(n22n). Between these two is the chasm of our ignorance. Anuj Dawar Complexity Theory Formalising Algorithms To prove a lower bound on the complexity of a problem, rather than a specic algorithm, we need to prove a statement about all algorithms for solving it. In order to prove facts about all algorithms, we need a mathematically precise denition of algorithm. We will use the Turing machine. The simplicity of the Turing machine means it's not useful for actually expressing algorithms, but very well suited for proofs about all algorithms. Anuj Dawar Complexity Theory Turing Machines For our purposes, a Turing Machine consists of: • Q a nite set of states; • Σ a nite set of symbols, including t and .. • s 2 Q an initial state; • δ :(Q × Σ) ! (Q [ facc; rejg) × Σ × fL; R; Sg A transition function that species, for each state and symbol a next state (or accept acc or reject rej), a symbol to overwrite the current symbol, and a direction for the tape head to move (L left, R right, or S - stationary) Anuj Dawar Complexity Theory Congurations A complete description of the conguration of a machine can be given if we know what state it is in, what are the contents of its tape, and what is the position of its head. This can be summed up in a simple triple: Denition A conguration is a triple (q; w; u), where q 2 Q and w; u 2 Σ? The intuition is that (q; w; u) represents a machine in state q with the string wu on its tape, and the head pointing at the last symbol in w. The conguration of a machine completely determines the future behaviour of the machine. Anuj Dawar Complexity Theory Computations Given a machine M = (Q; Σ; s; δ) we say that a conguration (q; w; u) yields in one step (q0; w 0; u0), written 0 0 0 (q; w; u) !M (q ; w ; u ) if • w = va ; • δ(q; a) = (q0; b; D); and • either D = L and w 0 = v and u0 = bu or D = S and w 0 = vb and u0 = u or D = R and w 0 = vbc and u0 = x, where u = cx. If u is empty, then w 0 = vbt and u0 is empty. Anuj Dawar Complexity Theory Computations The relation ? is the reexive and transitive closure of . !M !M A sequence of congurations c1;:::; cn, where for each i, ci !M ci+1, is called a computation of M. The language L(M) ⊆ Σ? accepted by the machine M is the set of strings ? acc for some and fx j (s; .; x) !M ( ; w; u) w ug A machine M is said to halt on input x if for some w and u, either ? acc or ? rej (s; .; x) !M ( ; w; u) (s; .; x) !M ( ; w; u) Anuj Dawar Complexity Theory Decidability A language L ⊆ Σ? is recursively enumerable if it is L(M) for some M. A language L is decidable if it is L(M) for some machine M which halts on every input. A language L is semi-decidable if it is recursively enumerable. A function f :Σ? ! Σ? is computable, if there is a machine M, such that for all , ? acc x (s; .; x) !M ( ;.f (x);") Anuj Dawar Complexity Theory Example Consider the machine with δ given by: . 0 1 t s s; .; R rej; 0; S rej; 1; S q; t; R q rej; .; R q; 1; R q; 1; R q0; 0; R q0 rej; .; R rej; 0; S q0; 1; L acc; t; S This machine, when started in conguration (s; .; t1n0) eventually halts in conguration (acc;. t 1n+10t;"). Anuj Dawar Complexity Theory Multi-Tape Machines The formalisation of Turing machines extends in a natural way to multi-tape machines. For instance a machine with k tapes is specied by: • Q, Σ, s; and • δ :(Q × Σk ) ! Q [ facc; rejg × (Σ × fL; R; Sg)k Similarly, a conguration is of the form: (q; w1; u1;:::; wk ; uk ) Anuj Dawar Complexity Theory Running Time With any Turing machine M, we associate a function r : IN ! IN called the running time of M. r(n) is dened to be the largest value R such that there is a string x of length n so that the computation of M starting with conguration (s; .; x) is of length R (i.e. has R successive congurations in it) and ends with an accepting conguration. In short, r(n) is the length of the longest accepting computation of M on an input of length n. Anuj Dawar Complexity Theory Complexity For any function f : IN ! IN, we say that a language L is in TIME(f ) if there is a machine M = (Q; Σ; s; δ), such that: • L = L(M); and • The running time of M is O(f ). Similarly, we dene SPACE(f ) to be the languages accepted by a machine which uses O(f (n)) tape cells on inputs of length n. In dening space complexity, we assume a machine M, which has a read-only input tape, and a separate work tape. We only count cells on the work tape towards the complexity. Anuj Dawar Complexity Theory Decidability and Complexity For every decidable language L, there is a computable function f such that L 2 TIME(f ) If L is a semi-decidable (but not decidable) language accepted by M, then there is no computable function f such that every accepting computation of M, on input of length n is of length at most f (n).