BU CS 332 – Theory of Computation

Lecture 22: • NP-Completeness Example Reading: • Space Complexity Sipser Ch 8.1-8.2 • Savitch’s Theorem

Mark Bun April 22, 2020 NP-completeness Definition: A language is NP-complete if 1) NP, and 𝐵𝐵 2) Every language NP is poly-time reducible to 𝐵𝐵 ∈ , i.e., (“ is NP-hard”) 𝐴𝐴 ∈ 𝐵𝐵 𝐴𝐴 ≤p 𝐵𝐵 𝐵𝐵

Theorem: If NP and for some NP-complete language , then is also NP-complete 𝐶𝐶 ∈ 𝐵𝐵 ≤p 𝐶𝐶 𝐵𝐵 𝐶𝐶

4/22/2020 CS332 - Theory of Computation 2 (3-CNF Satisfiability)

3𝑆𝑆Definition(s):𝑆𝑆𝑆𝑆 • A literal either a variable of its negation , • A clause is a disjunction (OR) of literals Ex. 𝑥𝑥5 𝑥𝑥7 • A 3-CNF is a conjunction (AND) of clauses where5 each7 2 clause contains exactly 3 literals 𝑥𝑥 ∨ 𝑥𝑥 ∨ 𝑥𝑥 Ex. … =

𝐶𝐶1 ∧ 𝐶𝐶2 ∧ ∧ 𝐶𝐶𝑚𝑚 5 7 2 3 4 1 1 1 1 =𝑥𝑥 ∨ 𝑥𝑥 ∨is𝑥𝑥a satisfiable∧ 𝑥𝑥 ∨ 𝑥𝑥 3∨ 𝑥𝑥 CNF∧ ⋯ ∧ 𝑥𝑥 ∨ 𝑥𝑥 ∨ 𝑥𝑥 Cook-Levin Theorem: is NP-complete 3𝑆𝑆𝑆𝑆𝑆𝑆 𝜑𝜑 𝜑𝜑 − 4/22/2020 CS332 - Theory of Computation 3 3𝑆𝑆𝑆𝑆𝑆𝑆 Some general reduction strategies • Reduction by simple equivalence Ex. and 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 − 𝑆𝑆𝑆𝑆𝑆𝑆 ≤p 𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉 − 𝐶𝐶𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂 𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉 − 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 ≤p 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 − 𝑆𝑆𝑆𝑆𝑆𝑆 • Reduction from special case to general case Ex.

𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉 − 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 ≤p 𝑆𝑆𝑆𝑆𝑆𝑆 − 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶

• Gadget reductions Ex.

3𝑆𝑆𝑆𝑆𝑆𝑆 ≤p 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 − 𝑆𝑆𝑆𝑆𝑆𝑆

4/22/2020 CS332 - Theory of Computation 4 Independent Set

An independent set in an undirected graph is a set of vertices that includes at most one endpoint of every edge. 𝐺𝐺

= , is an undirected graph containing an independent set with vertices} 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 − 𝑆𝑆𝑆𝑆𝑆𝑆 𝐺𝐺 𝑘𝑘 𝐺𝐺 ≥ 𝑘𝑘 • Is there an independent set of size ≥ 6? • Yes. independent set

• Is there an independent set of size ≥ 7? • No.

4/22/2020 CS332 - Theory of Computation 5 Independent Set is NP-complete 1) NP 2) Reduce 3 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 − 𝑆𝑆𝑆𝑆𝑆𝑆 ∈ 𝑆𝑆𝑆𝑆𝑆𝑆 ≤p 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 − 𝑆𝑆𝑆𝑆𝑆𝑆 Proof. “On input , where is a 3CNF formula, 1. Construct graph from 𝜑𝜑 𝜑𝜑 • contains 3 vertices for each clause, one for each literal. 𝐺𝐺 𝜑𝜑 • Connect 3 literals in a clause in a triangle. 𝐺𝐺 • Connect literal to each of its negations. 2. Output , , where is the number of clauses in .”

𝐺𝐺 𝑘𝑘 𝑘𝑘 𝜑𝜑

4/22/2020 CS332 - Theory of Computation 6 Example of the reduction =

𝜑𝜑 𝑥𝑥1 ∨ 𝑥𝑥2 ∨ 𝑥𝑥3 ∧ 𝑥𝑥1 ∨ 𝑥𝑥2 ∨ 𝑥𝑥3 ∧ 𝑥𝑥1 ∨ 𝑥𝑥2 ∨ 𝑥𝑥4

4/22/2020 CS332 - Theory of Computation 7 Proof of correctness for reduction

Let = # clauses and = # literals in Claim: is satisfiable iff has an ind. set of size 𝑘𝑘 𝑙𝑙 𝜑𝜑 Given𝜑𝜑 a satisfying assignment,𝐺𝐺 select one literal𝑘𝑘 from each triangle. This is an ind. set of size ⟹ Let be an ind. set of size 𝑘𝑘 • must contain exactly one vertex in each triangle ⟸ 𝑆𝑆 𝑘𝑘 • Set these literals to true, and set all other variables in an arbitrary way𝑆𝑆 • Truth assignment is consistent and all clauses satisfied

Runtime: ( + ) which is polynomial in input size 2 4/22/2020 CS332 - Theory of Computation 8 𝑂𝑂 𝑘𝑘 𝑙𝑙 Space Complexity

4/22/2020 CS332 - Theory of Computation 9 Complexity measures we’ve studied so far • Deterministic time TIME • Nondeterministic time NTIME • Classes P, NP

Many other resources of interest: Space (memory), randomness, parallel runtime / #processors, quantum entanglement, interaction, communication, …

4/22/2020 CS332 - Theory of Computation 10 Space analysis Space complexity of a TM (algorithm) = maximum number of tape cell it uses on a worst-case input

Formally: Let . A TM runs in space ( ) if on every input , halts on using at most cells 𝑓𝑓 ∶ ℕ∗ → ℕ 𝑀𝑀 𝑓𝑓 𝑛𝑛 𝑤𝑤 ∈ Σ 𝑀𝑀 𝑤𝑤 𝑓𝑓 𝑛𝑛 For nondeterministic machines: Let . An NTM runs in space ( ) if on every input , halts on using at most cells on every computational𝑓𝑓 ∶ ℕ →∗ ℕ branch 𝑁𝑁 𝑓𝑓 𝑛𝑛 𝑤𝑤 ∈ Σ 𝑁𝑁 𝑤𝑤 𝑓𝑓 𝑛𝑛

4/22/2020 CS332 - Theory of Computation 11 Space complexity classes

Let

A language𝑓𝑓 ∶ ℕ → ℕ SPACE( ( )) if there exists a basic single- tape (deterministic) TM that 1) Decides 𝐴𝐴, ∈and 𝑓𝑓 𝑛𝑛 2) Runs in space ( ( )𝑀𝑀) 𝐴𝐴 A language N𝑂𝑂SPACE𝑓𝑓 𝑛𝑛 ( ( )) if there exists a single- tape nondeterministic TM that 1) Decides 𝐴𝐴, ∈and 𝑓𝑓 𝑛𝑛 2) Runs in space ( ( )) 𝑁𝑁 𝐴𝐴 4/22/2020 CS332 - Theory of Computation 12 𝑂𝑂 𝑓𝑓 𝑛𝑛 Example: Space complexity of SAT Theorem: SPACE( ) Proof: The following deterministic TM decides using linear space𝑆𝑆𝑆𝑆𝑆𝑆 ∈ 𝑛𝑛 𝑆𝑆𝑆𝑆𝑆𝑆 On input where is a Boolean formula: 1. For each truth assignment to the variables 𝜑𝜑 𝜑𝜑 , … , of : 2. Evaluate on , … , 𝑥𝑥1 𝑥𝑥𝑚𝑚 𝜑𝜑 3. If any evaluation = 1, accept. Else, reject. 𝜑𝜑 𝑥𝑥1 𝑥𝑥𝑚𝑚

4/22/2020 CS332 - Theory of Computation 13 Example: NFA analysis Theorem: Let = is an NFA with = } Then NSPACE( ). ∗ 𝐴𝐴𝐴𝐴𝐴𝐴𝑁𝑁𝑁𝑁𝑁𝑁 𝐴𝐴 𝐴𝐴 𝐿𝐿 𝐴𝐴 Σ Proof: The following NTM decides in linear space 𝐴𝐴𝐴𝐴𝐴𝐴𝑁𝑁𝑁𝑁𝑁𝑁 ∈ 𝑛𝑛 𝐴𝐴𝐴𝐴𝐴𝐴𝑁𝑁𝑁𝑁𝑁𝑁 On input where is an NTM: 1. Place a marker on the start state of . 𝐴𝐴 𝐴𝐴 2. Repeat 2 times where is the # of states of : 𝐴𝐴 3. Nondeterministically𝑞𝑞 select . 𝑞𝑞 𝐴𝐴 4. Adjust the markers to simulate all ways for to read . 𝑎𝑎 ∈ Σ 5. Accept if at any point none of the markers are on an accept 𝐴𝐴 𝑎𝑎 state. Else, reject.

4/22/2020 CS332 - Theory of Computation 14 Example 0,1 ε 0 1 2 3 1

4/22/2020 CS332 - Theory of Computation 15 Space vs. Time

4/22/2020 CS332 - Theory of Computation 16 Space vs. Time

𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑓𝑓 𝑛𝑛 ⊆ 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝑓𝑓 𝑛𝑛 ⊆ 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑓𝑓 𝑛𝑛 How about the opposite direction? Can low-space algorithms be simulated by low-time algorithms?

4/22/2020 CS332 - Theory of Computation 17 Reminder: Configurations A configuration is a string where and , • Tape contents = (followed by blanks ) ∗ 𝑢𝑢𝑢𝑢𝑢𝑢 𝑞𝑞 ∈ 𝑄𝑄 𝑢𝑢 𝑣𝑣 ∈ Γ • Current state = 𝑢𝑢𝑢𝑢 ⊔ • Tape head on first symbol of 𝑞𝑞 𝑣𝑣 Example: 101 0111

5 Start configuration: 𝑞𝑞 Accepting configuration: = 𝑞𝑞0𝑤𝑤 Rejecting configuration: = 𝑞𝑞 𝑞𝑞accept 4/22/2020 CS332𝑞𝑞 - Theory𝑞𝑞 ofrejec Computationt 18 Reminder: Configurations Consider a TM with • states • tape alphabet {0, 1} 𝑘𝑘 • space ( ) How many configurations are possible when this TM is run on an input𝑓𝑓 𝑛𝑛 0,1 ? 𝑛𝑛 𝑤𝑤 ∈ Observation: If a TM enters the same configuration twice when run on input , it loops forever Corollary: A TM running in space ( ) also runs in time 2 𝑤𝑤 𝑂𝑂 𝑓𝑓 𝑛𝑛 𝑓𝑓 𝑛𝑛 4/22/2020 CS332 - Theory of Computation 19 Savitch’s Theorem

4/22/2020 CS332 - Theory of Computation 20 Savitch’s Theorem: Deterministic vs. Nondeterministic Space Theorem: Let be a function with . Then . 𝑓𝑓 𝑓𝑓2𝑛𝑛 ≥ 𝑛𝑛 Proof𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 idea:𝑓𝑓 𝑛𝑛 ⊆ 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑓𝑓 𝑛𝑛 • Let be an NTM deciding in space ( )

• We 𝑁𝑁construct a TM deciding𝑓𝑓 in space𝑓𝑓 𝑛𝑛 2 • Actually solve a more𝑀𝑀 general problem:𝑓𝑓 𝑂𝑂 𝑓𝑓 𝑛𝑛 • Given configurations , of and natural number , decide whether can go from to in steps on some nondeterministic𝑐𝑐1 𝑐𝑐2 path.𝑁𝑁 𝑡𝑡 1 2 • Procedure CANYIELD𝑁𝑁 ( , , ) 𝑐𝑐 𝑐𝑐 ≤ 𝑡𝑡

1 2 4/22/2020 CS332𝑐𝑐 - Theory𝑐𝑐 of Computation𝑡𝑡 21 Savitch’s Theorem Theorem: Let be a function with . Then . 𝑓𝑓 𝑓𝑓2𝑛𝑛 ≥ 𝑛𝑛 Proof𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 idea:𝑓𝑓 𝑛𝑛 ⊆ 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑓𝑓 𝑛𝑛 • Let be an NTM deciding in space ( ) = “On input : 𝑁𝑁 𝑓𝑓 𝑓𝑓 𝑛𝑛 1. Output the result of CANYIELD( , , 2 )” 𝑀𝑀 𝑤𝑤 𝑑𝑑𝑓𝑓 𝑛𝑛 𝑐𝑐1 𝑐𝑐2 Where CANYIELD( , , ) decides whether can go from configuration to in steps on some nondeterministic path 𝑐𝑐1 𝑐𝑐2 𝑡𝑡 𝑁𝑁 𝑐𝑐1 𝑐𝑐2 ≤ 𝑡𝑡

4/22/2020 CS332 - Theory of Computation 22 Savitch’s Theorem CANYIELD( , , ) decides whether can go from configuration to in steps on some nondeterministic path: 𝑐𝑐1 𝑐𝑐2 𝑡𝑡 𝑁𝑁 𝑐𝑐1 𝑐𝑐2 ≤ 𝑡𝑡 CANYIELD( , , ) = 1. If = 1, accept if = or yields in one transition. 𝑐𝑐1 𝑐𝑐2 𝑡𝑡 Else, reject. 𝑡𝑡 𝑐𝑐1 𝑐𝑐2 𝑐𝑐1 𝑐𝑐2 2. If > 1, then for each config of with cells: 3. Run CANYIELD( , , /2 ). 𝑡𝑡 𝑐𝑐𝑚𝑚𝑚𝑚𝑚𝑚 𝑁𝑁 ≤ 𝑓𝑓 𝑛𝑛 4. Run CANYIELD( , , /2 ). 〈𝑐𝑐1 𝑐𝑐𝑚𝑚𝑚𝑚𝑚𝑚 𝑡𝑡 〉 5. If both runs accept, accept. 〈𝑐𝑐𝑚𝑚𝑚𝑚𝑚𝑚 𝑐𝑐2 𝑡𝑡 〉 6. Reject.

4/22/2020 CS332 - Theory of Computation 23 Complexity class PSPACE

Definition: PSPACE is the class of languages decidable in polynomial space on a basic single-tape (deterministic) TM PSPACE = SPACE( ) ∞ 𝑘𝑘 ⋃𝑘𝑘=1 𝑛𝑛 Definition: NPSPACE is the class of languages decidable in polynomial space on a single-tape (nondeterministic) TM NPSPACE = NSPACE( ) ∞ 𝑘𝑘 ⋃𝑘𝑘=1 𝑛𝑛

4/22/2020 CS332 - Theory of Computation 24 Relationships between complexity classes 1. P NP PSPACE EXP since (2 ( )) ⊆ ⊆ ⊆ 𝑂𝑂 𝑓𝑓 𝑛𝑛 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑓𝑓 𝑛𝑛 ⊆ 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 EXP

2. P EXP (Monday) PSPACE=NPSPACE Which containments NP ≠ in (1) are proper? P Unknown!

4/22/2020 CS332 - Theory of Computation 25