Space Complexity Space Complexity Review Complexity Theory Space Complexity Daniel Borchmann, Markus Krötzsch Review Computational Logic 2015-11-25 cba cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #1 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #2 Space Complexity Space Complexity Space Complexity Space Complexity Review: Space Complexity Classes Recall our earlier definition of space complexities: Definition 10.1 Let f : N R+ be a function. → DSpace(f(n)) is the class of all languages for which there is an Space Complexity L O(f(n))-space bounded Turing machine deciding . L NSpace(f(n)) is the class of all languages for which there is an L O(f(n))-space bounded nondeterministic Turing machine deciding . L Being O(f(n))-space bounded requires a (nondeterministic) TM to halt on every input and to use f( w ) tape cells on every computation path. ≤ | | cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #3 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #4 Space Complexity Space Complexity Space Complexity Space Complexity Space Complexity Classes The Power Of Space Some important space complexity classes: Space seems to be more powerful than time because space can be reused. L = LogSpace = DSpace(log n) logarithmic space PSpace = DSpace(nd ) polynomial space Example 10.2 d 1 [≥ Sat can be solved in linear space: d ExpSpace = DSpace(2n ) exponential space Just iterate over all possible truth assignments (each linear in size) and d 1 [≥ check if one satisfies the formula. NL = NLogSpace = NSpace(log n) nondet. logarithmic space Example 10.3 NPSpace = NSpace(nd ) nondet. polynomial space Tautology can be solved in linear space: d 1 Just iterate over all possible truth assignments (each linear in size) and [≥ d check if all satisfy the formula. NExpSpace = NSpace(2n ) nondet. exponential space d 1 [≥ More generally: NP PSpace and coNP PSpace ⊆ ⊆ cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #5 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #6 Space Complexity Space Complexity Space Complexity Space Complexity Linear Compression Tape Reduction Theorem 10.5 Theorem 10.4 + For every functionf : N R allk 1 and Σ∗: For every functionf : N R+, for allc N, and for every f-space → ≥ L ⊆ → ∈ If can be decided by anf-space boundedk-tape Turing-machine, bounded (deterministic/nondeterminsitic) Turing machine : L M it can also be decided by anf-space bounded 1-tape Turing-machine there is a max 1, 1 f(n) -space bounded (deterministic/nondeterminsitic) { c } Turing machine that accepts the same language as . M0 M Proof idea. Proof idea. Combine tapes with a similar reduction as for time. Compress space to avoid linear increase. Similar to (but much simpler than) linear speed-up. This justifies using O-notation for defining space classes. Recall that we still use a separate read-only input tape to define some space complexities, such as LogSpace. cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #7 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #8 Space Complexity Space Complexity Space Complexity Space Complexity Time vs. Space Number of Possible Configurations Theorem 10.6 Let := (Q, Σ, Γ, q0, δ, qstart) be a 2-tape Turing machine M For all functionsf : N R+: (1 read-only input tape + 1 work tape) → DTime(f) DSpace(f) and NTime(f) NSpace(f) Recall: A configuration of is a quadruple (q, p1, p2, x) where ⊆ ⊆ M q Q is the current state, Proof. ∈ pi N is the head position on tape i, and Visiting a cell takes at least one time step. ∈ x Γ is the tape content. ∈ ∗ Theorem 10.7 Let w Σ be an input to and n := w . Then also p n. ∈ ∗ M | | 1 ≤ + For all functionsf : N R with f(n) log n: If is f(n)-space bounded we can assume p f(n) and x f(n) → ≥ M 2 ≤ | | ≤ ( ) ( ) DSpace(f) DTime(2O f ) and NSpace(f) DTime(2O f ) Hence, there are at most ⊆ ⊆ Q n f(n) Γ f(n) = n 2O(f(n)) = 2O(f(n)) Proof. | | · · · | | · Based on configuration graphs and a bound on the number of possible different configurations on inputs of length n configurations. (the last equality requires f(n) log n). ≥ cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #9 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #10 Proof. Space Complexity Space Complexity Space Complexity Space Complexity O(f(n)) ConfigurationBuild the configuration Graphs graph (time2 ) and find a path from the start Time vs. Space to an accepting stop configuration (time2 O(f(n))). The possible computations of a TM (on input w) form a directed graph: M Theorem 10.6 Vertices: configurations that can reach (on input w) For all functionsf : N R+: M → Edges: there is an edge from C1 to C2 if C1 C2 `M DTime(f) DSpace(f) and NTime(f) NSpace(f) (C2 reachable from C1 in a single step) ⊆ ⊆ This yields the configuration graph Proof. Could be infinite in general. Visiting a cell takes at least one time step. For f(n)-space bounded 2-tape TMs, there can be at most2 O(f(n)) vertices and2 (2O(f(n)))2 = 2O(f(n)) edges Theorem 10.7 · For all functionsf : N R+ with f(n) log n: A computation of on input w corresponds to a path in the configuration → ≥ M ( ) ( ) graph from the start configuration to a stop configuration. DSpace(f) DTime(2O f ) and NSpace(f) DTime(2O f ) ⊆ ⊆ Hence, to test if accepts input w, M Proof. construct the configuration graph and Based on configuration graphs and a bound on the number of possible find a path from the start to an accepting stop configuration. configurations. cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #11 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #12 Proof. Build the configuration graph (time2 O(f(n))) and find a path from the start to an accepting stop configuration (time2 O(f(n))). Space Complexity Space Complexity Space Complexity Space Complexity Basic Space/Time Relationships Nondeterminism in Space Most experts think that nondeterministic TMs can solve strictly more Applying the results of the previous slides, we get the following relations: problems when given the same amount of time than a deterministic TM: Most believe that P ( NP L NL P NP PSpace NPSpace ExpTime NExpTime ⊆ ⊆ ⊆ ⊆ ⊆ ⊆ ⊆ How about nondeterminism in space-bounded TMs? We also noted P coNP PSpace. ⊆ ⊆ Theorem 10.8 (Savitch’s Theorem, 1970) Open questions: For any functionf : N R+ withf (n) log n: What is the relationship between space classes and their co-classes? → ≥ NSpace(f(n)) DSpace(f 2(n)). What is the relationship between deterministic and non-deterministic ⊆ space classes? That is: nondeterminism adds almost no power to space-bounded TMs! cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #13 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #14 Space Complexity Space Complexity Space Complexity Space Complexity Consequences of Savitch’s Theorem Proving Savitch’s Theorem Savitch’s Theorem: For any function f : N R+ with f(n) log n: → ≥ Simulating nondeterminism with more space: Use configuration graph of nondeterministic space-bounded TM NSpace(f(n)) DSpace(f 2(n)). ⊆ Corollary 10.9 Check if an accepting configuration can be reached PSpace = NPSpace. Store only one computation path at a time (depth-first search) This still requires exponential space. We want quadratic space! Proof. What to do? PSpace NPSpace is clear. The converse follows since the square of a ⊆ polynomial is still a polynomial. Things we can do: Store one configuration: Similarly for “bigger” classes, e.g., ExpSpace = NExpSpace. one configuration requires log n + O(f(n)) space Corollary 10.10 if f(n) log n, then this is O(f(n)) space ≥ NL DSpace(O(log2 n)). Store log n configurations (remember we have log2 n space) ⊆ Iterate over all configurations (one by one) Note that log2(n) < O(log n), so we do not obtain NL = L from this. cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #15 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #16 Space Complexity Space Complexity Space Complexity Space Complexity Proving Savitch’s Theorem: Key Idea An Algorithm for Yieldability To find out if we can reach an accepting configuration, we solve a slighly 01 CanYield(C1,C2,k){ more general question: 02 if k = 1 : 03 return (C1 = C2) or (C1 C2) `M 04 else if k > 1 : Yieldability 05 for each configuration C of for input size n : Input: TM configurations C1 and C2, integer k M 06 if CanYield(C1,C,k/2) and Problem: Can TM get from C1 to C2 in at most k steps? 07 CanYield(C,C2,k/2): 08 return true Approach: check if there is an intermediate configuration C0 such that 09 // eventually, if no success: 10 return false (1) C can reach C0 in k/2 steps and 1 11} (2) C0 can reach C2 in k/2 steps { Deterministic: we can try all C0 (iteration) We only call CanYield only with k a power of 2, so k/2 N { Space-efficient: we can reuse the same space for both steps ∈ cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #17 cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #18 Space Complexity Space Complexity Space Complexity Space Complexity Space Requirement for the Algorithm Simulating Nondeterministic Space-Bounded TMs 01 CanYield(C ,C ,k){ 1 2 Input:TM that runs in NSpace(f(n)); input word w of length n 02 if k = 1 : M 03 return (C1 = C2) or (C1 C2) `M Algorithm: 04 else if k > 1 : 05 for each configuration C of for input size n : Modify to have a unique accepting configuration Caccept M