COMPLEXITY THEORY Review Lecture 12: Hierarchy Theorems Markus Krotzsch¨ Knowledge-Based Systems TU Dresden, 29th Nov 2017 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 2 of 22 Relationships of Complexity Classes Relationships of Complexity Classes Relating different complexity classes is a central goal of complexity theory Complexity classes differ by: L NL P NP PSpace = NPSpace Exp Underlying machine model (e.g., DTM vs. NTM) ⊆ ⊆ ⊆ ⊆ ⊆ • = = = ? = = = Restricted resource (e.g., time or space) • coL coNL coP coNP coPSpace = coNPSpace coExp Resource bound (e.g., polynomial or exponential) ⊆ ⊆ ⊆ ⊆ ⊆ • Some intuitions: Nondeterminism seems to add some more power • Obvious question: Space seems to be more powerful than time • More resources seem to add more power • Are any of these strict (? ⊆ However: it is often difficult to confirm these intuitive ideas formally (and many of them remain unproven) Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 3 of 22 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 4 of 22 Something we do know At least one thing is known to be (mostly) true: Intuition: If we allow a given Turing machine to use strictly more time or space, then it can really solve strictly more problems. Diagonalisation This is not always true, but it is true for “reasonable” ways of defining resource • bounds, such as polynomial or exponential We will formalise and prove it later today • The proof method we will use is called diagonalisation • Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 5 of 22 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 6 of 22 Review: Cantor’s Argument Review: The Halting Problem Diagonalisation is the basis of a well known argument to show that the powerset 2S of a We have used a similar argument to show undecidability of the Halting problem: countable set is not countable S Proof: Suppose for a contradiction that Halting is decidable. Proof: Suppose for a contradiction that 2S is countable. Then set of all Turing machines can be enumerated in a list , , , ... • M1 M2 M3 Then the sets in 2 can be enumerated in a list , , , ... We are interested in their halting on inputs of the form for some TM • S S1 S2 S3 ⊆ S • hMi M Let us write this list as boolean matrix with rows representing the sets We can write it as a boolean matrix with rows representing the TMs • • , , , ..., columns representing a (countably infinite) enumeration of , and 1, 2, 3, ..., columns representing a (countably infinite) enumeration of strings S1 S2 S3 S M M M boolean entries encoding the relationship. , and boolean entries encoding if TM halts. ∈ hMi For a contradiction, define a set by diagonalisation to differ from all other in Using a decider for the halting problem, we can define a TM d by diagonalisation • Sd Si • M the enumeration: to differ from all other i in the enumeration: M s s s ... ... 1 2 3 hM1i hM2i hM3i ... ... S1 × M1 × ... ... S2 × M2 × 3 ... 3 ... S. ×. ×. M. ×. ×. .. .. ... ... Sd × × Md × × Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 7 of 22 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 8 of 22 Generalising Diagonalisation Diagonalisation of Resources Consider resources R and R for two classes of Turing machines and , and two To generalise diagonalisation as amethod for complexity classes, we consider arbitrary 1 2 K1 K2 functions f , f : N N. resources (time, space, . ): 1 2 → Definition 12.3: We say that R1(f1) allows diagonalisation over R2(f2) if there ex- Definition 12.1: Given a class of Turing machines (e.g., 2-tape determinis- K ists a Turing machine 1 such that tic TMs), R is a resource (e.g., time or space) defined for all machines in if D ∈ K K L( ) R (f ), and R (w) N for all and all words w. • D ∈ 1 1 M ∈ ∪ {∞} M ∈ K for each that is R -bounded by f , there exists a w such that • M ∈ K2 2 2 Then, for any function f : N N gives rises to a class of languages: , w L( ) if and only if , w < L( ). → hM i ∈ D hM i M R(f ) = L there is with L( ) = L and R (w) f ( w ) for all w Σ∗ . { | M ∈ K M M ≤ | | ∈ } Theorem 12.4: If R (f ) allows diagonalisation over R (f ), then R (f ) R (f ) , . 1 1 2 2 1 1 \ 2 2 ∅ Proof: Let be as in Definition 12.3. We show L( ) < R (f ). Example 12.2: We will use, e.g., the following resources: D D 2 2 (1) Suppose for a contradiction that there is 2 that is R2-bounded by f2 with DTime time used by a deterministic 1-tape TM M ∈ K • L( ) = L( ). DTime time used by a deterministic k-tape TM D M • k (2) We obtain a contradiction, since, by Definition 12.3, there is a word w such that DTime time used by a deterministic TM with any number of tapes , , • ∗ w L( ) = L( ) if and only if w < L( ) hM i ∈ D M hM i M Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 9 of 22 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 10 of 22 Reasonable bounds What kind of functions should we consider as resource bounds? Functions f : N N can be very weird • → The intuition “higher bound more power” turns out to be wrong in general • ⇒ However, our intuition can be confirmed for “reasonable” functions: Hierarchy Theorems Definition 12.5: A function t : N N is time-constructible if t(n) n for all n and → ≥ there exists a TM that computes t(n) in unary in time O(t(n)). A function s : N N is space-constructible if s(n) log n and there exists a TM → ≥ that computes s(n) in unary in space O(s(n)). Note 1: We do consider arbitrary deterministic multi-tape TMs here. Note 2: A TM that computes f (n) “in unary” takes n as input and writes a symbol (say x) f (n) times before terminating Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 11 of 22 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 12 of 22 Time and space constructible functions Using Constructibility to Halt We had required time-bounded nondeterministic TMs to halt on all computation paths, There are alternative definitions of time and space constructibility in the literature, but even if not accepting. — Is this really necessary? the general intuition is similar: Time-constructible: Computing f does not require significantly more time than the Theorem 12.7: Given a time-constructible function f and an NTM , one can • M resulting value of f construct an O(f )-time bounded NTM 0 that accepts exactly those words w that M | | Space-constructible: Computing f does not require significantly more space than accepts in f ( w ) steps. • M | | the resulting value of f Consequences: (1) we can enforce timely termination on unsuccessful paths; (2) if we All functions commonly used to bound time or space satisfy these criteria: have at least polynomial time, this can also be achieved with only one tape. Proof: On input w, operates as follows: Theorem 12.6: If f and g are time-constructible (space-constructible), then so M0 are f + g, f g, 2f , and f g. Moreover, the following common functions have these (1) Compute f ( w ) on a separate tape (creating f ( w ) symbols). This can be done in · | | | | properties: O(f ( w )) time. | | nd (d 1), bn (b 2), and n! are time-constructible (2) Perform the same transitions as (on dedicated tapes) while “counting down” the • ≥ ≥ M d n f ( w ) symbols in each step log n, n (d 1), b (b 2), and n! are space-constructible | | • ≥ ≥ (3) Terminate if either terminates (in this case return its result) or if the countdown M reaches 0 (in this case reject). Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 13 of 22 Markus Krötzsch, 29th Nov 2017 Complexity Theory slide 14 of 22 Doing more by using more tapes Doing more by using more tapes (2) We first show a preliminary result: “more time + more tapes = more power” We first show a preliminary result: “more time + more tapes = more power” Theorem 12.8: Let f , g : N N such that f is time-constructible, and g o(f ). Theorem 12.8: Let f , g : N N such that f is time-constructible, and g o(f ). → ∈ → ∈ Then, for all k N, we have Then, for all k N, we have ∈ ∈ DTimek(g) ( DTime (f ) DTimek(g) ( DTime (f ) ∗ ∗ Proof: Clearly, DTimek(g) DTimek(f ) DTime (f ). We get DTime (f ) , DTimek(g) by Proof (continued): To invoke Theorem 12.4, we still have to show that, for every k-tape ⊆ ⊆ ∗ ∗ showing that that DTime (f ) allows diagonalisation over DTimek(g). TM that is g-time bounded, there is a word w such that ∗ M We define a multi-tape TM for inputs of the form , w (other cases do not matter): , w L( ) if and only if , w < L( ) , D hM i hM i ∈ D hM i M Compute f ( w ) in unary on a separate “countdown” tape For this, we need to show that there is a word w for which ’s simulation of will • |hM i|, D M Simulate on w , using an appropriate number of tapes (see Theorem 3.8). terminate on time: • M hM i , Time-bound the simulation by f ( w ) using the countdown tape as in For all , there is a constant number c of steps that will at most need to • |hM i| • M M D Theorem 12.7 simulate one step of (this depends on the size of ) M M If rejects (in this time bound), then accept; otherwise, if accepts or fails to Since g o(f ) there is a number n0 such that f (n) c g(n) for all n n0.