NP Completeness COMP 455 – 002, Spring 2019
Jim Anderson (modified by Nathan Otterness) 1 Time Complexity (Revisited)
A problem is a yes/no question. Example: The clique problem: Given a graph 퐺 and integer 푘, does 퐺 contain a 푘-clique? ❖(A 풌-clique is a fully-connected group of 푘 nodes.) 퐺, 푘 is an instance of this problem (for some specific graph 퐺). A problem is decidable if an algorithm exists that will take any instance of this problem and always produce the correct yes/no answer. We will only be concerned with decidable problems when An algorithm is a TM that always halts. discussing NP completeness. Jim Anderson (modified by Nathan Otterness) 2 Another Example Problem
Consider this undirected graph: 푣1 Let 푘 = 3. We can encode an instance of the 푣2 clique problem as a string 푤: 푣4
푤 = 3 1,2 1,4 2,3 2,4 3,4 3,5 4,5 푣3 푣5
The language corresponding to the clique problem could be stated as follows:
퐿 = {all strings 푘 푖1, 푖2 … 푖푚, 푗푚 | the corresponding graph has a 푘-clique}. If 푀 is a TM that solves the clique problem, then 퐿 푀 = 퐿. Jim Anderson (modified by Nathan Otterness) 3 Additional Constraints on Encoding Many different ways exist to encode problems. For these problems, the choice of encoding matters! The standard encoding for these problems is binary or something “polynomially related” to binary. ❖ For example, decimal is OK. Unary is not OK. There are two reasons for this: With unary encodings, polynomial-time solutions exist to some NP-complete problems. This is because time complexity is measured based on input length! Using artificially long inputs is “cheating”. Real computers use binary encodings.
Jim Anderson (modified by Nathan Otterness) 4 Two Basic Complexity Classes
(There are lots of other classes—we’re just focusing on these two for now.) P-Time: Any language 퐿 accepted in polynomial time by some deterministic TM. NP-Time: Any language 퐿 accepted in polynomial time by some nondeterministic TM. ❖ Recall that the time complexity of a NDTM is based on the smallest number of moves needed to accept a string of a given length.
We usually abbreviate P-Time as “P” and NP-Time as “NP.”
Jim Anderson (modified by Nathan Otterness) 5 The Relationship Between P and NP
P
NP
Every DTM is also a NDTM, so P is in NP. It’s not known whether 푃 = 푁푃 or 푃 ⊂ 푁푃.
Jim Anderson (modified by Nathan Otterness) 6 P vs. NP The P = NP question is the most important open question in theoretical computer science. Here’s why: Lots of interesting problems are in NP. Problems in P have “efficient” solutions. (Meaning deterministic, polynomial-time solutions.) For many important problems in NP (e.g. yes/no counterparts to optimization problems or graph problems) we haven’t found “efficient” solutions despite decades of research. ❖ The suggests that 푃 ≠ 푁푃. ❖ But, if 푃 = 푁푃, then these problems do have “efficient” solutions.
Jim Anderson (modified by Nathan Otterness) 7 Hard and Complete Problems
We want to identify the “hardest” problems in NP. If the “hard” problems have deterministic polynomial- time solutions, then so do all problems in NP. This reduces the P=NP question to that of whether one of these “hard” problems is in P.
Jim Anderson (modified by Nathan Otterness) 8 In other words, we Hard and Complete Problems want a DTM that converts instances of one problem to What do we specifically mean by “hard” and instances of another “complete”? We need some more definitions: in polynomial time. 퐿 is polynomially transformable to 퐿′ if there exists a polynomial-time-bounded DTM 푀 that can convert each string 푤 in the alphabet of 퐿 into a string 푤′ in the alphabet of 퐿′ such that 푤 ∈ 퐿 if and only if 푤′ ∈ 퐿′. Let 퐶 be a class of languages. 퐿 is hard for 푪 if every language in 퐶 is polynomially transformable to 퐿. 퐿 is complete for 푪 if 퐿 is in 퐶 and hard for 퐶.
Jim Anderson (modified by Nathan Otterness) 9 NP-Complete Problems
Using the previous definitions, 퐿 is NP-complete if it is in NP and NP-hard. Note: Showing that 퐿 is in NP is usually straightforward: show that a NDTM can accept 퐿 by “guessing” an answer and then verifying the guess is correct in polynomial time. Our goal is to first identify some NP-complete problems.
Jim Anderson (modified by Nathan Otterness) 10 NP-Complete Problems
Given the unlikelihood of P=NP, if a language 퐿 is NP- complete, then 퐿 is probably an intractable problem. ❖Note: “NP-complete” is not (currently) synonymous with “exponential time.” We can show P=NP if we’re able to find a single NP- Complete problem that’s in P. Nobody’s been able to do so, but it hasn’t been proven impossible yet, either.
Jim Anderson (modified by Nathan Otterness) 11 “Guess-and-Verify” Example
Let a NDTM accept strings of the form 10푖110푖2 … 10푖푘 such that there is some 퐼 ⊆ 1, … , 푘 for which This technically isn’t a σ푗∈퐼 푖푗 = σ푗∉퐼 푖푗. “legal” encoding (it (This is called the partition problem.) encodes 푖푛 using a unary # of 0s). We’re Here’s how our NDTM can work: just using it here to The TM has three tapes, the input is on tape 1. simplify the example. The TM scans the input and copies each group of 0s to either tape 2 or tape 3, the choice being nondeterministic. The TM accepts if an equal number of 0s were copied to both tapes after scanning the entire input string.
Jim Anderson (modified by Nathan Otterness) 12 Proving a Problem is NP-Complete
There are two ways to show 퐿 is NP complete: Prove it from scratch (very difficult!) Use reduction: I. Show that 퐿 is in NP (i.e. design a “guess-and- verify” polynomial-time NDTM for 퐿), and II. Show that a known NP-complete problem can be polynomially transformed to 퐿. So, if we want to use reduction, we first need a known NP-complete problem.
Jim Anderson (modified by Nathan Otterness) 13 Cook’s Theorem
The satisfiability problem is the following: given a Boolean expression 퐸, is it possible to assign values to the variables in such a way that makes 퐸 true? Example: 푥 ∨ 푦 ∧ 푧 is satisfiable. 푥 ∨ 푦 ∧ 푥ҧ ∧ 푦ത is not satisfiable. Cook’s Theorem: The satisfiability problem is NP- complete. The proof is too long to cover in this class, so we will need to take it on faith.
Jim Anderson (modified by Nathan Otterness) 14 Variations on Satisfiability A conjunct is a group of A literal is either 푥 or ¬푥, where 푥 is a Boolean literals or clauses that variable. are “and”-ed together. A Boolean expression is in conjunctive normal form (CNF) if it is a conjunct of “clauses” of literals: ❖Example: 푎 ∨ 푏 ∧ 푐 ∧ ¬푎 ∨ 푏 ∨ ¬푐 k-CNF: A conjunct of clauses containing 푘 literals. A clause is a group k-satisfiability: Satisfiability for expressions in k- of literals that are CNF. “or”-d together.
Jim Anderson (modified by Nathan Otterness) 15 Variations on Satisfiability
We will show the following:
Satisfiability We won’t cover this in class (see Section 10.3.3 of the book for details). Satisfiability for expressions in These are easier to CNF use in subsequent We will cover this reductions. polynomial transformation. 3-satisifiability
Jim Anderson (modified by Nathan Otterness) 16 3-Sat
The satisfiability problem for CNF expressions with exactly three literals per clause is called 3- satisifiability (“3-Sat”). ❖Example: Is 푎 ∨ 푏 ∨ 푐 ∧ ¬푐 ∨ 푑 ∨ 푒 satisfiable?
Jim Anderson (modified by Nathan Otterness) 17 3-Sat
Theorem 10.15: 3-satisfiability is NP-complete. Proof: We must do two things: Show 3-Sat is in NP. ❖ This is easy: just nondeterministically generate an assignment for each variable, then, in polynomial time verify that each clause evaluates to true. This is called a reduction Show 3-Sat is NP-hard. from CNF-Sat. ❖ To do this we will show that if we can solve 3-Sat efficiently, then we can solve CNF-Sat efficiently.
Jim Anderson (modified by Nathan Otterness) 18 Reducing from CNF-Sat to 3-Sat
Our goal is to take a Boolean expression in CNF and, in polynomial time, generate a new expression in 3-CNF that is satisfiable if and only if the original CNF expression is satisfiable. We want to show that if a polynomial time solution to 3-CNF exists, then the following algorithm would be possible: 1. Convert a CNF-Sat problem to a 3-Sat problem in polynomial time. 2. Use the polynomial-time 3-Sat algorithm to determine if the converted expression is satisfiable.
Jim Anderson (modified by Nathan Otterness) 19 Reducing from CNF-Sat to 3-Sat
Converting from CNF-Sat to 3-Sat: We can convert each clause containing 푛 literals 푥1 … 푥푛 to a conjunct of 푛 − 2 new clauses, each of which contain exactly three literals.
❖This requires adding 푛 − 3 new literals 푦1 … 푦푛−3 per clause. Example:
푥1 ∨ 푥2 ∨ ⋯ ∨ 푥푛 becomes: 푥1 ∨ 푥2 ∨ 푦1 ∧ 푦1 ∨ 푥3 ∨ 푦2 ∧ 푦2 ∨ 푥4 ∨ 푦3 ∧ ⋯ ∧ 푦푛−4 ∨ 푥푛−2 ∨ 푦푛−3 ∧ 푦푛−3 ∨ 푥푛−1 ∨ 푥푛
Jim Anderson (modified by Nathan Otterness) 20 Reducing from CNF-Sat to 3-Sat We want to show that the old expression is satisfiable if and only if the new expression is satisfiable. Old: 푥 ∨ 푥 ∨ ⋯ ∨ 푥 First: The old expression is satisfiable ⇒ the new 1 2 푛 expression is satisfiable. New: ❖Assume the old expression is satisfiable 푥1 ∨ 푥2 ∨ 푦1 ∧ 푦1 ∨ 푥3 ∨ 푦2 ∧ ❖⇒ ∃ an assignment s.t. the old expression is true 푦2 ∨ 푥4 ∨ 푦3 ∧ ⋯ ∧ 푦 ∨ 푥 ∨ 푥 ❖⇒ some 푥푖 is true in the assignment 푛−3 푛−1 푛
❖⇒ Setting 푦푗 ≔ true for 푗 ≤ 푖 − 2 and 푦푗 ≔ false for 푗 < 푖 − 2 makes the new expression true. ❖⇒ The new expression is satisfiable.
Jim Anderson (modified by Nathan Otterness) 21 Reducing from CNF-Sat to 3-Sat Old: 푥1 ∨ 푥2 ∨ ⋯ ∨ 푥푛
Second: The new expression is satisfiable ⇒ the old New: expression is satisfiable. 푥1 ∨ 푥2 ∨ 푦1 ∧ 푦 ∨ 푥 ∨ 푦 ∧ Assume an assignment exists such that the new 1 3 2 푦2 ∨ 푥4 ∨ 푦3 ∧ ⋯ ∧ expression is true (i.e. it’s satisfiable). One of the 푦 ∨ 푥 ∨ 푥 following must be the case: 푛−3 푛−1 푛
❖ If 푦1 is false, then 푥1 or 푥2 is true.
❖ If 푦푛−3 is true, then 푥푛−1 or 푥푛 is true.
❖ If 푦1 is true and 푦푛−3 is false, then some 푖 exists such that 푦푖 is true and 푦푖+1 is false. This implies that 푥푖+2 is true. For all of the above cases, the old expression also had to be true because some 푥푖 was true.
Jim Anderson (modified by Nathan Otterness) 22 Problems We Will Cover
CNF- Satisfiability 3- Clique Satisfiability Problem
Vertex Cover
Directed Hamiltonian I didn’t have time to do Circuit these, sorry… Undirected Hamiltonian Circuit Jim Anderson (modified by Nathan Otterness) 23 To show it’s in NP: 1. Nondeterministically The Clique Problem “guess” a selection of 푘 nodes in the graph. Theorem: The clique problem is NP-complete. 2. In polynomial time, check whether the 푘 nodes are fully connected. Proof: The clique problem is in NP. To show the clique problem is NP-hard we will show that CNF-Sat reduces to the clique problem.
Consider a CNF expression 퐹 = 퐹1 ∧ 퐹2 ∧ ⋯ ∧ 퐹푞, where
퐹푖 = 푥푖1 ∨ 푥푖2 ∨ ⋯ ∨ 푥푖푘푖 . We will construct a graph 퐺 that has a 푞-clique if and only if 퐹 is satisfiable.
Jim Anderson (modified by Nathan Otterness) 24 The Clique Problem
퐹 = 퐹1 ∧ 퐹2 ∧ ⋯ ∧ 퐹푞, where 퐹푖 = 푥푖1 ∨ 푥푖2 ∨ ⋯ ∨ 푥푖푘푖 . Construct the graph 퐺 = 푉, 퐸 as follows: ❖Each pair 푖, 푗 is a vertex, where 1 ≤ 푖 ≤ 푞 and 1 ≤ 푗 ≤ 푘푖. ❖Each 푖, 푗 , 푘, 푙 is an edge, where 푖 ≠ 푘 and 푥푖푗 ≠ 푥푘푙. ❖Given 퐹, you can construct 퐺 in polynomial time.
Jim Anderson (modified by Nathan Otterness) 25 The Clique Problem
Example: 퐹 = 푦1 ∨ 푦2 ∧ 푦2 ∨ 푦3 ∧ 푦3 ∨ 푦1 The literals are:
푥11 = 푦1 푥21 = 푦2 푥31 = 푦3
푥12 = 푦2 푥22 = 푦3 푥32 = 푦1
풚ퟏ 풚ퟐ 풚ퟑ [1,1] [2,1] [3,1] 퐺: [1,2] [2,2] [3,2] 풚ퟐ 풚ퟑ 풚ퟏ
Jim Anderson (modified by Nathan Otterness) 26 The Clique Problem
Example: 퐹 = 푦1 ∨ 푦2 ∧ 푦2 ∨ 푦3 ∧ 푦3 ∨ 푦1
풚ퟏ 풚ퟐ 풚ퟑ [1,1] [2,1] [3,1] 퐺: [1,2] [2,2] [3,2] 풚ퟐ 풚ퟑ 풚ퟏ
퐺 has a clique of size 푞 if and only if 퐹 is satisfiable. 퐺 has two 3-cliques: 1,1 , 2,1 , 3,1 corresponding to 푦1 = 푦2 = 푦3 = true. 1,2 , 2,2 , 3,2 corresponding to 푦1 = 푦2 = 푦3 = false.
Jim Anderson (modified by Nathan Otterness) 27 The Clique Problem Proof that 퐺 has a 푞-clique if and only if 퐹 is satisfiable: Assume 퐹 is satisfiable. ❖Then, ∃ an assignment to variables s.t. 퐹 is true.
❖Each 퐹푖 has ≥ 1 literal that is true. ❖ Let 푥푖푚푖 in 퐹푖 be true. ❖Consider 푖, 푚푖 | 1 ≤ 푖 ≤ 푞 .
❖Consider 푖, 푚푖 and 푗, 푚푗 where 푖 ≠ 푗. ❖ 푥푖푚푖 = 푥푗푚푗 = true implies 푥푖푚푖 ≠ 푥푗푚푗.
❖Since 푖 ≠ 푗, 푖, 푚푖 , [푗, 푚푗] is an edge.
❖So, 푖, 푚푖 | 1 ≤ 푖 ≤ 푞 is a 푞-clique.
Jim Anderson (modified by Nathan Otterness) 28 The Clique Problem Assume 퐺 has a 푞-clique.
❖Let the vertices in the clique be 푖, 푚푖 | 1 ≤ 푖 ≤ 푞 ❖ Let 푆푡푟푢푒 = 푦 | 푥푖푚푖 = 푦 for some 푖 and
푆푓푎푙푠푒 = 푦 | 푥푖푚푖 = 푦ത for some 푖 .
❖By setting each variable in 푆푡푟푢푒 to true and each variable in 푆푓푎푙푠푒 to false, each clause 퐹푖 is made true. So, 퐹 is satisfiable.
Jim Anderson (modified by Nathan Otterness) 29 Vertex Cover Theorem 10.20: The Vertex Cover problem is NP- Complete.
Vertex Cover: Given a graph 퐺 = 푉, 퐸 , find a subset 푆 of the vertices 푉 such that each edge in 퐸 is incident upon some vertex in 푆. (This is also called node cover.)
The Vertex Cover Problem: Does an undirected graph 퐺 have a vertex cover of size 푘?
Jim Anderson (modified by Nathan Otterness) 30 Vertex Cover Vertex Cover example: Does this graph 퐺 contain a vertex cover of size 2?
1
5 2
4 3
Yes: 2, 3 is a vertex cover.
Jim Anderson (modified by Nathan Otterness) 31 Vertex Cover To prove that Vertex Cover is in NP: 1. Nondeterministically “Guess” a set of 푘 vertices in 퐺. 2. In polynomial time, check to see if this set is a vertex cover. To prove that Vertex Cover is NP-hard, we will reduce from the clique problem We show that a polynomial-time solution to vertex cover will let us solve the clique problem in polynomial time.
Jim Anderson (modified by Nathan Otterness) 32 Vertex Cover Here’s the transformation that we’ll make: 1. Start with an instance of the clique problem: 퐺, 푘 ❖ Let 퐺 = 푉, 퐸 2. In polynomial time, compute 퐺ҧ ❖ Let 퐺ҧ = 푉, 퐸ത , where 퐸ത = { 푣, 푤 | 푣 ≠ 푤 ∧ 푣, 푤 ∉ 퐸ሽ 3. Construct the following instance of the vertex cover problem: 퐺ҧ, 푉 − 푘 . 퐺ҧ has a vertex cover of size 푉 − 푘 if and only if 퐺 had a clique of size 푘. (We’ll prove this on the following slides.)
Jim Anderson (modified by Nathan Otterness) 33 Vertex Cover Transformation example: Input to the clique problem: Does 퐺 have a 3-clique? 퐺 = 1 1. Start with an instance of (Yes, it does) the clique problem: 퐺, 푘 1 ❖ 퐺 = 푉, 퐸 5 2 2. In polynomial time, 5 compute 퐺ҧ ❖ 퐺ҧ = 푉, 퐸ത , where 퐸ത = { 푣, 푤 | 푣 ≠ 푤 ∧ 4 4 3 푣, 푤 ∉ 퐸ሽ 3. Construct the following instance of the vertex cover problem: 퐺ҧ, 푉 − 푘 .
Jim Anderson (modified by Nathan Otterness) 34 Vertex Cover Transformation example: Next, construct 퐺ҧ:
1. Start with an instance of the clique problem: 퐺, 푘 1 1 ❖ 퐺 = 푉, 퐸 퐺ҧ 2. In polynomial time, 퐺: : compute 퐺ҧ ҧ ത 5 2 5 2 ❖ 퐺 = 푉, 퐸 , where 퐸ത = { 푣, 푤 | 푣 ≠ 푤 ∧ 푣, 푤 ∉ 퐸ሽ 3. Construct the following 4 3 4 3 instance of the vertex cover problem: 퐺ҧ, 푉 − 푘 .
Jim Anderson (modified by Nathan Otterness) 35 Vertex Cover Transformation example: Does 퐺ҧ contain a vertex cover of size 푉 − 푘?
❖(푘 was from the size of the clique in the clique 1. Start with an instance of problem, so 푉 − 푘 is 2.) the clique problem: 퐺, 푘 ❖ 퐺 = 푉, 퐸 1 2. In polynomial time, (Yes, it does) ҧ 퐺ҧ compute 퐺 : ❖ 퐺ҧ = 푉, 퐸ത , where 5 2 2 퐸ത = { 푣, 푤 | 푣 ≠ 푤 ∧ 푣, 푤 ∉ 퐸ሽ 3. Construct the following instance of the vertex 4 3 3 cover problem: 퐺ҧ, 푉 − 푘 .
Jim Anderson (modified by Nathan Otterness) 36 Vertex Cover
Now, we need to prove that 퐺ҧ has a vertex cover of size 푉 − 푘 if and only if 퐺 has a clique of size 푘: Assume 푆 is a clique in 퐺. ❖No edge in 퐺ҧ connects two vertices in 푆, and ❖Every edge in 퐺ҧ is incident upon at least one vertex in 푉 − 푆, so ❖푉 − 푆 is a vertex cover of 퐺ҧ.
Jim Anderson (modified by Nathan Otterness) 37 Vertex Cover
Now, we need to prove that 퐺ҧ has a vertex cover of size 푉 − 푘 if and only if 퐺 has a clique of size 푘: Assume 푉 − 푆 is a vertex cover of 퐺ҧ ❖This means every edge in 퐺ҧ is incident upon at least one vertex in 푉 − 푆, and ❖No edge in 퐺ҧ connects two vertices in 푆, so ❖Every pair of vertices in 푆 is connected in 퐺 ❑This is the definition of a clique, so 푆 is a clique in 퐺.
Jim Anderson (modified by Nathan Otterness) 38