Introduction to Computational Complexity A 10-lectures Graduate Course Martin Stigge, [email protected] Uppsala University, Sweden 13.7. - 17.7.2009 Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 1 / 148 Introduction Administrative Meta-Information 5-Day course, Monday (13.7.) to Friday (17.7.) Schedule: Mon (13.7.): 10:00 - 12:00, 16:30 - 18:30 Tue (14.7.): 10:00 - 12:00, 14:00 - 16:00 Wed (15.7.): 10:00 - 12:00, 14:00 - 16:00 Thu (16.7.): 10:00 - 12:00, 14:00 - 16:00 Fri (17.7.): 10:00 - 12:00, 16:30 - 18:30 Lecture notes avaiable at: http://www.it.uu.se/katalog/marst984/cc-st09 Some small assignments at end of day Course credits: ?? Course is interactive, so: Any questions so far? Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 2 / 148 Introduction What is Computational Complexity? Studies intrinsic complexity of computational tasks Absolute Questions: I How much time is needed to perform the task? I How much resources will be needed? Relative Questions: I More difficult than other tasks? I Are there \most difficult” tasks? (Surprisingly: Many relative answers, only few absolute ones..) Rigorous treatment: I Mathematical formalisms, minimize \hand-waving" I Precise definitions I Theorems have to be proved After all: Complexity Theory Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 3 / 148 Introduction What is Computational Complexity? (Cont.) Basis: Computability Theory I Provides models of computation I Explores their strength (expressiveness) I Question: \What can be computed (at all)?" Then: Complexity Theory I Tries to find meaningful complexity measures I Tries to classify and relate problems I Tries to find upper and lower complexity bounds I Question: \What can efficiently be computed?" One core concern: I What does “efficiently” actually mean? ? I Proving vs. Verifying (P = NP problem) Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 4 / 148 Introduction Course Outline 0 Introduction 1 Basic Computability Theory Formal Languages Model of Computation: Turing Machines Decidability, Undecidability, Semi-Decidability 2 Complexity Classes Landau Symbols: The O(·) Notation Time and Space Complexity Relations between Complexity Classes 3 Feasible Computations: P vs. NP Proving vs. Verifying Reductions, Hardness, Completeness This is a theoretical course Natural NP-complete problems { expect a lot of \math"! 4 Advanced Complexity Concepts Non-uniform Complexity Probabilistic Complexity Classes Interactive Proof Systems Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 5 / 148 Basic Computability Theory Formal Languages Problems as Formal Languages Start with very high-level model of computation Assume a machine with input and output Formal notation for format: I Σ = fσ1; : : : ; σk g is a finite set of symbols I w = (w1;:::; wl ) is a word over Σ: 8i : wi 2 Σ F Write also just w1w2 ::: wl I l is the length of w, also denoted jwj I " is the empty word, i.e., j"j = 0 k I Σ is the set of words of length k ∗ S k I Σ = k≥0 Σ are all words over Σ ∗ I A language is a set L ⊆ Σ ∗ I Let L1; L2 ⊆ Σ , language operations: F L1 [ L2 (union), L1 \ L2 (intersection), L1 − L2 (difference) ∗ F L := Σ − L (complement) F L1L2 := fw j 9w1 2 L1; w2 2 L2 : w = w1w2g (concatenation) Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 6 / 148 Basic Computability Theory Formal Languages Problems as Formal Languages (Example) Example: NAT Let Σ := f0; 1;:::; 9g Σ∗ is all strings with digits Let [n]10 denote decimal representation of n 2 N ∗ NAT := f[n]10 j n 2 Ng ( Σ all representations of naturals ∗ I 010 2 Σ − NAT Machine for calculating square: ∗ Input: w = [n]10 2 Σ ∗ 2 Output: v 2 Σ with v = [n ]10 (Plus error-handling for w 2= NAT) Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 7 / 148 Basic Computability Theory Formal Languages Problems as Formal Languages (2nd Example) Example: PRIMES Let Σ := f0; 1;:::; 9g and NAT as before PRIMES := f[p]10 j p is a prime numberg Clearly: PRIMES ( NAT Machine M for checking primality: ∗ Input: w = [n]10 2 Σ Output: 1 if n is prime, 0 otherwise This is a decision problem: I PRIMES are the positive instances, ∗ I Σ − PRIMES (everything else) the negative instances I M has to distinguish both (it decides PRIMES) Important concept, will come back to that later! Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 8 / 148 Basic Computability Theory Model of Computation: Turing Machines Model of Computation Input/Output format defined. What else? Model of Computation should: I Define what we mean with \computation", \actions", ... I Be simple and easy to use I ... but yet powerful Models: Recursive Functions, Rewriting Systems, Turing Machines, ... All equally powerful: Church's Thesis All \solvable" problems can be solved by any of the above formalisms. We will focus on the Turing Machine. Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 9 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine Turing Machines are like simplified computers containing: I A tape to read/write on F Contains squares with one symbol each F Is used for input, output and temporary storage F Unbounded I A read/write head F Can change the symbol on the tape at current position F Moves step by step in either direction I A finite state machine F Including an initial state and final states Looks simple, but is very powerful Standard model for the rest of the course Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 10 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine: Definition Definition (Turing Machine) A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F ) where Q is a finite set of states; Γ is the tape alphabet including the blank: 2 2 Γ; q0 is the initial state, q0 2 Q; F is the set of final states, F ⊆ Q; δ is the transition function, δ :(Q − F ) × Γ ! Q × Γ × fR; N; Lg: Operation: Start in state q0, input w is on tape, head over its first symbol Each step: I Read current state q and symbol a at current position I Lookup δ(q; a) = (p; b; D) I Change to state p, write b, move according to D Stop as soon as q 2 F . Left on tape: Output Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 11 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine: Configuration Configuration (w; q; v) denotes status after each step: I Tape contains wv (with infinitely many 2 around) I Head is over first symbol of v I Machine is in state q Start configuration: ("; q0; w) if input is w End configuration: (v; q; z) for a q 2 F I Output is z, denoted by M(w) I In case machine doesn't halt (!): M(w) =% Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 12 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine: Step Relation Step relation: Formalizes semantics of Turing machine Definition (Step Relation) ∗ Let M = (Q; Γ; δ; q0; F ), define ` for all w; v 2 Γ ; a; b 2 Γ and q 2 Q as: 8 (wac; p; v) if δ(q; b) = (p; c; R); <> (wa; q; bv) ` (wa; p; cv) if δ(q; b) = (p; c; N); :>(w; p; acv) if δ(q; b) = (p; c; L): α reaches β in 1 step: α ` β α reaches β in k steps: α `k β α reaches β in any number of steps: α `∗ β Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 13 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine: The Universal Machine Turing machine model is quite simple Can be easily simulated by a human I Provided enough pencils, tape space and patience Important result: Machines can simulate machines I Turing machines are finite objects! I Effective encoding into words over an alphabet I Also configurations are finite! Encode them also Simulator machine U only needs to I Receive an encoded M as input I Input of M is w, give that also to U I U maintains encoded configurations of M and applies steps Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 14 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine: The Universal Machine (Cont.) Let hMi be encoding of machine M. Theorem (The Universal Machine) There exists a universal Turing machine U, such that for all Turing machines M and all words w 2 Σ∗: U(hMi; w) = M(w) In particular, U does not halt iff 1 M does not halt. (Without proof.) 1\if and only if" Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7. - 17.7.2009 15 / 148 Basic Computability Theory Model of Computation: Turing Machines Turing Machine: Transducers and Acceptors Definition so far: Receive input, compute output We call this a transducer: ∗ ∗ I Interpret a TM M as a function f :Σ ! Σ I All such f are called computable functions I Partial functions may be undefined for some inputs w F In case M does not halt for them (M(w) =%) I Total functions are defined for all inputs For decision problems L: Only want a positive or negative answer We call this an acceptor: I Interpret M as halting in F Either state qyes for positive instances w 2 L F Or in state qno for negative instances w 2= L I Output does not matter, only final state I M accepts the language L(M): ∗ ∗ ∗ L(M) := fw 2 Σ j 9y; z 2 Γ :("; q0; w) ` (y; qyes ; z)g Rest of the course: Mostly acceptors Martin Stigge (Uppsala University, SE) Computational Complexity Course 13.7.