Chapter 21

Reductions and NP

CS 473: Fundamental , Spring 2011 April 14, 2011

21.1 Reductions Continued

21.1.0.1 Polynomial Time A polynomial time reduction from a X to a decision problem Y is an A that has the following properties:

• given an instance IX of X, A produces an instance IY of Y

•A runs in time polynomial in |IX |. This implies that |IY | (size of IY ) is polynomial in |IX |

• Answer to IX YES iff answer to IY is YES.

Notation: X ≤P Y if X reduces to Y

Proposition 21.1.1 If X ≤P Y then a polynomial time algorithm for Y implies a polyno- mial time algorithm for X. Such a reduction is called a Karp reduction. Most reductions we will need are Karp reductions

21.1.0.2 A More General Reduction Turing Reduction (the one given in the book)

Problem X polynomial time reduces to Y if there is an algorithm A for X that has the following properties:

1 • on any given instance IX of X, A uses polynomial in |IX | “steps” • a step is either a standard computation step or

• a sub-routine call to an algorithm that solves Y Note: In making sub-routine call to algorithm to solve Y , A can only ask questions of size polynomial in |IX |. Why? Above reduction is called a Turing reduction.

21.1.0.3 Example of Turing Reduction Input Collection of arcs on a circle.

Goal Compute the maximum number of non-overlapping arcs.

Reduced to the following problem:? Input Collection of intervals on the line.

Goal Compute the maximum number of non-overlapping intervals. How? Used algorithm for interval problem multiple times.

21.1.0.4 Turing vs Karp Reductions • Turing reductions more general than Karp reductions

• Turing reduction useful in obtaining algorithms via reductions

• Karp reduction is simpler and easier to use to prove hardness of problems

• Perhaps surprisingly, Karp reductions, although limited, suffice for most known NP- Completeness proofs

21.1.1 The Satisfiability Problem (SAT) 21.1.1.1 Propositional Formulas

Definition 21.1.2 Consider a set of boolean variables x1, x2, . . . xn

• A literal is either a boolean variable xi or its negation ¬xi

• A clause is a disjunction of literals. For example, x1 ∨ x2 ∨ ¬x4 is a clause • A formula in conjunctive normal form (CNF) is propositional formula which is a conjunction of clauses

– (x1 ∨ x2 ∨ ¬x4) ∧ (x2 ∨ ¬x3) ∧ x5 is a formula in CNF

2 • A formula ϕ is in 3CNF if it is a CNF formula such that every clause has exactly 3 literals

– (x1∨x2∨¬x4)∧(x2∨¬x3∨x1) is a 3CNF formula, but (x1∨x2∨¬x4)∧(x2∨¬x3)∧x5 is not.

21.1.1.2 Satisfiability

SAT Given a CNF formula ϕ, is there a truth assignment to variables such that ϕ evaluates to true?

Example 21.1.3 (x1 ∨ x2 ∨ ¬x4) ∧ (x2 ∨ ¬x3) ∧ x5 is satisfiable; take x1, x2, . . . x5 to be all true (x1 ∨ ¬x2) ∧ (¬x1 ∨ x2) ∧ (¬x1 ∨ ¬x2) ∧ (x1 ∨ x2) is not satisfiable

3-SAT Given a 3CNF formula ϕ, is there a truth assignment to variables such that ϕ evaluates to true?

21.1.1.3 Importance of SAT and 3-SAT • SAT and 3-SAT are basic constraint satisfaction problems • Many different problems can reduced to them because of the simple yet powerful ex- pressivity of logical constraints • Arise naturally in many applications involving hardware and software verification and correctness • As we will see, it is a fundamental problem in theory of NP-Completeness

21.1.2 Sat and 3-SAT

21.1.2.1 SAT ≤P 3-SAT

Easy to see that 3-SAT ≤P SAT. A 3-SAT instance is also an instance of SAT.

We can show that SAT ≤P 3-SAT.

Given ϕ a SAT formula we create a 3-SAT formula ϕ0 such that • ϕ is satisfiable iff ϕ0 is satisfiable • ϕ0 can be constructed from ϕ in time polynomial in |ϕ|. Idea: if a clause of ϕ is not of length 3, replace it with several clauses of length exactly 3

3 21.1.2.2 SAT ≤P 3-SAT

Reduction Ideas Challenge: Some of the clauses in ϕ may have less or more than 3 literals. For each clause with < 3 or > 3 literals, we will construct a set of logically equivalent clauses. • Case clause with 1 literal: Let c = `. Let u, v be new variables. Consider c0 = (` ∨ u ∨ v) ∧ (` ∨ u ∨ ¬v) ∧ (` ∨ ¬u ∨ v) ∧ (` ∨ ¬u ∨ ¬v) Observe that c0 is satisfiable iff c is satisfiable

21.1.2.3 SAT ≤P 3-SAT (contd)

Reduction Ideas: 2 and more literals

• Case clause with 2 literals: Let c = `1 ∨ `2. Let u be a new variable. Consider

0 c = (`1 ∨ `2 ∨ u) ∧ (`1 ∨ `2 ∨ ¬u) Again c is satisfiable iff c0 is satisfiable

21.1.2.4 SAT ≤P 3-SAT (contd)

Reduction Ideas: 2 and more literals

• Case clause with > 3 literals: Let c = `1 ∨ · · · ∨ `k. Let u1, . . . uk−3 be new variables. Consider 0 c = (`1 ∨ `2 ∨ u1) ∧ (`3 ∨ ¬u1 ∨ u2) ∧ (`4 ∨ ¬u2 ∨ u3) ∧ · · · ∧ (`k−2 ∨ ¬uk−4 ∨ uk−3) ∧ (`k−1 ∨ `k ∨ ¬uk−3) c is satisfiable iff c0 is satisfiable

Another way to see it — reduce size of clause by one:

0 c = (`1 ∨ `2 ... ∨ `k−2 ∨ uk−3) ∧ (`k−1 ∨ `k ∨ ¬uk−3)

21.1.2.5 An Example

Example 21.1.4 ϕ = (¬x1 ∨ ¬x4) ∧ (x1 ∨ ¬x2 ∨ ¬x3) ∧ (¬x2 ∨ ¬x3 ∨ x4 ∨ x1) ∧ (x1).

ψ = (¬x1 ∨ ¬x4 ∨ z) ∧ (¬x1 ∨ ¬x4 ∨ ¬z)

∧ (x1 ∨ ¬x2 ∨ ¬x3)

∧ (¬x2 ∨ ¬x3 ∨ y1) ∧ (x4 ∨ x1 ∨ ¬y1)

∧ (x1 ∨ u ∨ v) ∧ (x1 ∨ u ∨ ¬v) ∧ (x1 ∨ ¬u ∨ v) ∧ (x1 ∨ ¬u ∨ ¬v)

4 21.1.2.6 Overall Reduction Algorithm

Input: CNF formula ϕ for each clause c of ϕ if c does not have exactly 3 literals construct c’ as before else c’ = c ψ is conjunction of all c’ constructed in loop is ψ satisfiable?

Correctness (informal) ϕ is satisfiable iff ψ is satisfiable because for each clause c, the new 3CNF formula c0 is logically equivalent to c.

21.1.2.7 What about 2-SAT? 2-SAT can be solved in polynomial time! No known polynomial time reduction from SAT (or 3-SAT) to 2-SAT. If there was, then SAT and 3-SAT would be solvable in polynomial time.

21.1.3 3-SAT and Independent Set

21.1.3.1 3-SAT ≤P Independent Set Input Given a 3CNF formula ϕ

Goal Construct a graph Gϕ and number k such that Gϕ has an independent set of size k iff ϕ is satisfiable. Gϕ should be constructible in time polynomial in size of ϕ

Importance of reduction: Although 3-SAT is much more expressive, it can be reduced to a seemingly specialized Independent Set problem.

21.1.3.2 Interpreting 3-SAT There are two ways to think about 3-SAT

• Find a way to assign 0/1 (false/true) to the variables such that the formula evaluates to true, that is each clause evaluates to true

• Pick a literal from each clause and find a truth assignment to make all of them true. You will fail if two of the literals you pick are in conflict, i.e., you pick xi and ¬xi

We will take the second view of 3-SAT to construct the reduction.

5 ¬x1 ¬x2 ¬x1

x2 x3 x1 x3 x2 x4

Figure 21.1: Graph for ϕ = (¬x1 ∨ x2 ∨ x3) ∧ (x1 ∨ ¬x2 ∨ x3) ∧ (¬x1 ∨ x2 ∨ x4)

21.1.3.3 The Reduction

• Gϕ will have one vertex for each literal in a clause • Connect the 3 literals in a clause to form a triangle; the independent set will pick at most one vertex from each clause, which will correspond to the literal to be set to true • Connect 2 vertices if they label complementary literals; this ensures that the literals corresponding to the independent set do not have a conflict • Take k to be the number of clauses

21.1.3.4 Correctness

Proposition 21.1.5 ϕ is satisfiable iff Gϕ has an independent set of size k (= number of clauses in ϕ).

Proof : ⇒ Let a be the truth assignment satisfying ϕ – Pick one of the vertices, corresponding to true literals under a, from each triangle. This is an independent set of the appropriate size

21.1.3.5 Correctness (contd)

Proposition 21.1.6 ϕ is satisfiable iff Gϕ has an independent set of size k (= number of clauses in ϕ).

Proof : ⇐ Let S be an independent set of size k – S must contain exactly one vertex from each clause – S cannot contain vertices labeled by conflicting clauses – Thus, it is possible to obtain a truth assignment that makes in the literals in S true; such an assignment satisfies one literal in every clause

6 21.1.3.6 Transitivity of Reductions

X ≤P Y and Y ≤P Z implies that X ≤P Z. Note: X ≤P Y does not imply that Y ≤P X and hence it is very important to know the FROM and TO in a reduction.

To prove X ≤P Y you need to show a reduction FROM X TO Y In other words show that an algorithm for Y implies an algorithm for X.

21.2 Definition of NP

21.2.0.7 Recap ...

Problems • Independent Set • Vertex Cover • Set Cover • SAT • 3-SAT

Relationship ≤P 3-SAT ≤P Independent Set ≥P Vertex Cover ≤P Set Cover 3-SAT ≤P SAT ≤P 3-SAT

21.3 Preliminaries

21.3.1 Problems and Algorithms 21.3.1.1 Problems and Algorithms: Formal Approach

Decision Problems • Problem Instance: Binary string s, with size |s| • Problem: A set X of strings on which the answer should be “yes”; we call these YES instances of X. Strings not in X are NO instances of X.

Definition 21.3.1 • A is an algorithm for problem X if A(s) = ”yes” iff s ∈ X • A is said to have a polynomial running time if there is a polynomial p(·) such that for every string s, A(s) terminates in at most O(p(|s|)) steps

7 21.3.1.2 Polynomial Time Definition 21.3.2 Polynomial time (denoted P ) is the class of all (decision) problems that have an algorithm that solves it in polynomial time

Example 21.3.3 ¡2-¿ Problems in P include

• Is there a shortest path from s to t of length ≤ k in G?

• Is there a flow of value ≥ k in network G?

• Is there an assignment to variables to satisfy given linear constraints?

21.3.1.3 Efficiency Hypothesis A problem X has an efficient algorithm iff X ∈ P , that is X has a polynomial time algorithm. Justifications:

• robustness of definition to variations in machines

• a sound theoretical definition

• most known polynomial time algorithms for “natural” problems have small polynomial running times

21.3.1.4 Problems with no known polynomial time algorithms

Problems

• Independent Set

• Vertex Cover

• Set Cover

• SAT

• 3-SAT

There are of course undecidable problems (no algorithm at all!) but many problems that we want to solve are like above. Question: What is common to above problems?

8 21.3.1.5 Efficient Checkability

Above problems share the following feature:

For any YES instance IX of X there is a proof/certificate/solution that is of length poly(|IX |) such that given a proof one can efficiently check that IX is indeed a YES instance Examples:

• SAT formula ϕ: proof is a satisfying assignment

• Independent Set in graph G and k: a subset S of vertices

21.3.2 Certifiers/Verifiers 21.3.2.1 Certifiers

Definition 21.3.4 An algorithm C(·, ·) is a certifier for problem X if for every s ∈ X there is some string t such that C(s, t) =”yes”, and conversely, if for some s and t, C(s, t) =”yes” then s ∈ X. The string t is called a certificate or proof for s

Efficient Certifier C is an efficient certifier for problem X if there is a polynomial p(·) such that for every string s, s ∈ X iff there is a string t with |t| ≤ p(|s|), C(s, t) = ”yes” and C runs in polynomial time

21.3.2.2 Example: Independent Set

• Problem: Does G = (V,E) have an independent set of size ≥ k?

– Certificate: Set S ⊆ V – Certifier: Check |S| ≥ k and no pair of vertices in S is connected by an edge

21.3.3 Examples 21.3.3.1 Example: Vertex Cover

• Problem: Does G have a vertex cover of size ≤ k?

– Certificate: S ⊆ V – Certifier: Check |S| ≤ k and that for every edge at least one endpoint is in S

9 21.3.3.2 Example: SAT • Problem: Does formula ϕ have a satisfying truth assignment?

– Certificate: Assignment a of 0/1 values to each variable – Certifier: Check each clause under a and say “yes” if all clauses are true

21.3.3.3 Example:Composites • Problem: Is number s a composite?

– Certificate: A factor t ≤ s such that t 6= 1 and t 6= s – Certifier: Check that t divides s (Euclid’s algorithm)

21.4 NP

21.4.1 Definition 21.4.1.1 Nondeterministic Polynomial Time Definition 21.4.1 Nondeterministic Polynomial Time (denoted by NP ) is the class of all problems that have efficient certifiers

Example 21.4.2 ¡2-¿ Independent Set, Vertex Cover, Set Cover, SAT, 3-SAT, Composites are all examples of problems in NP

21.4.1.2 Asymmetry in Definition of NP Note that only YES instances have a short proof/certificate. NO instances need not have a short certificate.

Example: SAT formula ϕ. No easy way to prove that ϕ is NOT satisfiable!

More on this and co-NP later on.

21.4.2 Intractibility 21.4.2.1 P versus NP Proposition 21.4.3 P ⊆ NP For a problem in P no need for a certificate!

Proof : Consider problem X ∈ P with algorithm A. Need to demonstrate that X has an efficient certifier

10 • Certifier C on input s, t, runs A(s) and returns the answer

• C runs in polynomial time

• If s ∈ X then for every t, C(s, t) = ”yes”

• If s 6∈ X then for every t, C(s, t) = ”no”

21.4.2.2 Exponential Time Definition 21.4.4 Exponential Time (denoted EXP ) is the collection of all problems that have an algorithm which on input s runs in exponential time, i.e., O(2poly(|s|))

Example: O(2n), O(2n log n), O(2n3 ), ...

21.4.2.3 NP versus EXP Proposition 21.4.5 NP ⊆ EXP

Proof : Let X ∈ NP with certifier C. Need to design an exponential time algorithm for X

• For every t, with |t| ≤ p(|s|) run C(s, t); answer “yes” if any one of these calls returns “yes”

• The above algorithm correctly solves X (exercise)

• Algorithm runs in O(q(|s| + |p(s)|)2p(|s|)), where q is the running time of C

21.4.2.4 Examples • SAT: try all possible truth assignment to variables

• Independent set: try all possible subsets of vertices

• Vertex cover: try all possible subsets of vertices

21.4.2.5 Is NP efficiently solvable? We know P ⊆ NP ⊆ EXP Big Question Is there are problem in NP that does not belong to P ? Is P = NP ?

11