Computational Complexity; slides 6, HT 2021 Strong NP-hardness, NP vs. co-NP, FNP

Prof. Paul W. Goldberg (Dept. of Computer Science, University of Oxford)

HT 2021

Paul Goldberg strong NP-hardness, co-NP, FNP 1 / 1 A Polynomial Time Algorithm for Knapsack?

Knapsack solved in time O(n`) using dynamic programming

Initialisation: Let M be a( ` + 1) × (n + 1) matrix entries of first row and column (row 0, col 0) of M are set to 0. Computation: For i = 1,..., n and w = 1 . . . `:

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Here, if w − wi < 0 we always take M(w, i − 1). M(w, i): Largest total value obtainable by selecting from the first i items with weight limit w Acceptance: If M contains an entry ≥ t, answer yes, otherwise no

Paul Goldberg strong NP-hardness, co-NP, FNP 2 / 1 0 0 0 0 0 0 1 3 3 3 0 1 4 4 4 0 1 4 4 5 0 1 4 7 7 0 1 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 1 2 3 4 5 Set M(w, 0) = M(0, i)=0 for all1 ≤ w ≤ ` 1 ≤ i ≤ n

Paul Goldberg strong NP-hardness, co-NP, FNP 3 / 1 1 3 3 3 1 4 4 4 1 4 4 5 1 4 7 7 1 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 2 0 3 0 4 0 5 0 Set M(w, 0) = M(0, i)=0 for all1 ≤ w ≤ ` 1 ≤ i ≤ n

Paul Goldberg strong NP-hardness, co-NP, FNP 3 / 1 1 3 3 3 1 4 4 4 1 4 4 5 1 4 7 7 1 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 2 0 3 0 4 0 5 0 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 3 3 3 4 4 4 4 4 5 4 7 7 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 2 0 1 3 0 1 4 0 1 5 0 1 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 3 3 4 4 4 4 4 5 4 7 7 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 3 2 0 1 3 0 1 4 0 1 5 0 1 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 3 3 4 4 4 4 5 4 7 7 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 3 2 0 1 4 3 0 1 4 0 1 5 0 1 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 3 3 4 4 4 5 4 7 7 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 3 2 0 1 4 3 0 1 4 4 0 1 5 0 1 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 3 3 4 4 4 5 7 7 4 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 3 2 0 1 4 3 0 1 4 4 0 1 4 5 0 1 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 3 3 4 4 4 5 7 7 8 8

Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 3 2 0 1 4 3 0 1 4 4 0 1 4 5 0 1 4 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 Example

Input: I := {1, 2, 3, 4} with

Values: v1 := 1 v2 := 3 v3 := 4 v4 := 2 Weight: w1 := 1 w2 := 1 w3 := 3 w4 := 2 Weight limit: ` := 5 Target value: t := 7

weight max. total value from first i items limit w i = 0 i = 1 i = 2 i = 3 i = 4 0 0 0 0 0 0 1 0 1 3 3 3 2 0 1 4 4 4 3 0 1 4 4 5 4 0 1 4 7 7 5 0 1 4 8 8 For i = 1,..., n set M(w, i) as

M(w, i) := max{M(w, i − 1), M(w − wi , i) + vi }

Paul Goldberg strong NP-hardness, co-NP, FNP 4 / 1 Strong NP-completeness: Problems which remain NP-complete even if all numbers are bounded by a polynomial in the input length (equivalently, for unary encoding of numbers).

Pseudo-Polynomial Time

This algorithm does not show that Knapsack is in P! The length of the input to Knapsack is O(n log `) n · ` is not bounded by a polynomial in the input length!

Pseudo-Polynomial Time: Algorithms polynomial in the maximum of the input length and the value of numbers occurring in the input.

If Knapsack is restricted to instances with ` ≤ p(n) for some polynomial p, then we obtain a problem in P.

Equivalently: Knapsack is in polynomial time for unary encoding of numbers.

Paul Goldberg strong NP-hardness, co-NP, FNP 5 / 1 Pseudo-Polynomial Time

This algorithm does not show that Knapsack is in P! The length of the input to Knapsack is O(n log `) n · ` is not bounded by a polynomial in the input length!

Pseudo-Polynomial Time: Algorithms polynomial in the maximum of the input length and the value of numbers occurring in the input.

If Knapsack is restricted to instances with ` ≤ p(n) for some polynomial p, then we obtain a problem in P.

Equivalently: Knapsack is in polynomial time for unary encoding of numbers.

Strong NP-completeness: Problems which remain NP-complete even if all numbers are bounded by a polynomial in the input length (equivalently, for unary encoding of numbers).

Paul Goldberg strong NP-hardness, co-NP, FNP 5 / 1 Strong NP-completeness

Pseudo Polynomial time: Algorithms polynomial in the maximum of the input length and the value of numbers occurring in the input. Examples. Subset Sum, Knapsack (In some contexts, provides a practical algorithm for NP-hard problem)

Strong NP-completeness: Problems which remain NP-complete even if all numbers are bounded by a polynomial in the input length. Examples. Clique, Vertex Cover (For a practical algorithm, look for some other special feature of the problem)

Paul Goldberg strong NP-hardness, co-NP, FNP 6 / 1 NP and co-NP

Notation. For a language L ⊆ Σ∗ let L := Σ∗ \L be its complement.

Definition. If C is a complexity class, we define

co-C := {L : L ∈ C}.

co-NP: In particular, co-NP := {L : L ∈ NP}

A problem belongs to co-NP, if no-instances have short certificates.

Paul Goldberg strong NP-hardness, co-NP, FNP 7 / 1 Definition. A language C ∈ co-NP is co-NP-complete, if L ≤p C for all L ∈ co-NP. Tautology is co-NP-complete: any reduction from an NP problem to SAT can convert to a reduction from a co-NP problem to Tautology co-NP-complete decision problems are as hard as NP-complete ones.

co-NP-completeness

Examples of problems in co-NP:

No Hamiltonian Cycle Given: Graph G Question: Is it true that G contains no Hamiltonian cycle?

Tautology Given: Formula ϕ Question: Is ϕ a tautology, i.e. satisfied by all assignments?

Paul Goldberg strong NP-hardness, co-NP, FNP 8 / 1 co-NP-completeness

Examples of problems in co-NP:

No Hamiltonian Cycle Given: Graph G Question: Is it true that G contains no Hamiltonian cycle?

Tautology Given: Formula ϕ Question: Is ϕ a tautology, i.e. satisfied by all assignments?

Definition. A language C ∈ co-NP is co-NP-complete, if L ≤p C for all L ∈ co-NP. Tautology is co-NP-complete: any reduction from an NP problem to SAT can convert to a reduction from a co-NP problem to Tautology co-NP-complete decision problems are as hard as NP-complete ones. Paul Goldberg strong NP-hardness, co-NP, FNP 8 / 1 P, NP, and co-NP

Proposition. 1 P = co-P 2 Hence, P ⊆ NP ∩ co-NP

Question: NP = co-NP? Most people do not think so (c.f. research in propositional proof theory).

P = NP ∩ co-NP? Again, most people do not think so.

Later: Ladner’s theorem: assuming P6=NP, there are “NP-intermediate” problems.

Paul Goldberg strong NP-hardness, co-NP, FNP 9 / 1 Search versus decision

Problem 1: boolean formula ϕ — is ϕ satisfiable?

Problem 2: Given a formula ϕ, find a satisfiable assignment, or answer “no”.

Problem 2 is at least as hard. But — we can say: “problem 2 is no harder than NP”; solve problem 2 with oracle for problem 1. “oracle”: an imaginary black-box that supports queries to a computational problem: given an input, will (in one step) tell you correct output.

Paul Goldberg strong NP-hardness, co-NP, FNP 10 / 1 FNP, reducing search to decision

FNP: problems of computing a function that can be checked in polynomial time: find a certificate, not just answer “yes” An FNP problem comprises a polynomially balanced relation R for which R(x, y) can be checked in time polynomial in |x|, |y|. Given x, search for y with R(x, y). Note: it’s asking too much to solve a NP search problem X using a single decision oracle. But can solve X using multiple oracle calls to corresponding decision problem.

Paul Goldberg strong NP-hardness, co-NP, FNP 11 / 1 Reducibility amongst FNP problems

Informally, X ≤p Y means: given an oracle for problem Y , can reconstruct a solution to X . In detail, reduction needs 2 functions f , g, where f maps instances of X to instances of Y , and g maps solutions of Y to solutions of X . If X and Y correspond with relations R1 and R2 respectively, want

(x, g(z)) ∈ R1 iff (f (x), z) ∈ R2.

I’ll come back to this in the final lecture. Meanwhile, think about how to compare Factoring in base-2 with Factoring in base-10.

Paul Goldberg strong NP-hardness, co-NP, FNP 12 / 1 Optimisation

“Is there a k-clique” is (in a sense) equally hard as “find a k-clique”

“What’s the size of the largest clique?” is (in a sense) harder! If we told the answer is some value k, an NP machine can verify k-clique(s) exist Need also a co-NP machine to verify: no k + 1-clique exists!

NP, or co-NP alone, don’t seem to be sufficient, more later.

In exercises later, will make a start at classifying problems like this

Paul Goldberg strong NP-hardness, co-NP, FNP 13 / 1