Computational Complexity

Home , CC (complexity)

Lecture 2 in which we talk about NP-completeness (reductions, reductions)

1 Recap

2 Recap

Languages in NP are of the form:

2 Recap

Languages in NP are of the form:

L= { x | ∃w, |w| < poly(|x|) s.t. (x,w) ∈ L’ }, where L’ i s i n P

2 Recap

Languages in NP are of the form:

L= { x | ∃w, |w| < poly(|x|) s.t. (x,w) ∈ L’ }, where L’ i s i n P

Today: Hardest problems in NP

2 Reductions

3 Reductions

At the heart of today’s complexity theory

3 Reductions

At the heart of today’s complexity theory

L1 ! L2 if problem of deciding L1 “reduces to that of deciding” L2

3 Reductions

At the heart of today’s complexity theory

L1 ! L2 if problem of deciding L1 “reduces to that of deciding” L2

if can decide L2, can decide L1

3 Turing and Many-One

4 Turing and Many-One

Turing re duction:

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One:

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One:

ML1 can query OL2 only once, and must output what OL2 outputs

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One:

ML1 can query OL2 only once, and must output what OL2 outputs

ML1 maps its input x to an input f(x) for OL2

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One:

ML1 can query OL2 only once, and must output what OL2 outputs

ML1 maps its input x to an input f(x) for OL2

x ∈ L1 ⇒ f(x) ∈ L2 and x ! L1 ⇒ f(x) ! L2

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One: L1 ML1 can query OL2 only once, and L2 must output what OL2 outputs

ML1 maps its input x to an input f(x) for OL2

x ∈ L1 ⇒ f(x) ∈ L2 and x ! L1 ⇒ f(x) ! L2

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One: L1 ML1 can query OL2 only once, and L2 must output what OL2 outputs

ML1 maps its input x to an input f(x) for OL2

x ∈ L1 ⇒ f(x) ∈ L2 and x ! L1 ⇒ f(x) ! L2

4 Turing and Many-One

Turing re duction:

Build a TM (oracle machine) ML1, s.t. using the oracle OL2 which decides L2, ML1^OL2 decides L1

ML1 may query OL2 many times (with different inputs)

Many-One: L1 ML1 can query OL2 only once, and L2 must output what OL2 outputs

ML1 maps its input x to an input f(x) for OL2

x ∈ L1 ⇒ f(x) ∈ L2 and x ! L1 ⇒ f(x) ! L2

4 Polynomial-Time Reduction

5 Polynomial-Time Reduction

Many-one reduction, where ML1 runs in polynomial time

5 Polynomial-Time Reduction

Many-one reduction, where ML1 runs in polynomial time

L1 !p L2

5 Polynomial-Time Reduction

Many-one reduction, where ML1 runs in polynomial time

L1 !p L2

L2 is “computationally (almost) as hard or harder” compared to L1

5 Polynomial-Time Reduction

Many-one reduction, where ML1 runs in polynomial time

L1 !p L2

L2 is “computationally (almost) as hard or harder” compared to L1

“almost”: reduction overheads (reduction time, size blow-up)

5 Polynomial-Time Reduction

Many-one reduction, where ML1 runs in polynomial time

L1 !p L2

L2 is “computationally (almost) as hard or harder” compared to L1

“almost”: reduction overheads (reduction time, size blow-up)

L2 may be way harder

5 Cook, Karp, Levin

6 Cook, Karp, Levin

Polynomial-time reduction

6 Cook, Karp, Levin

Polynomial-time reduction Cook: Turing reduction

6 Cook, Karp, Levin

Polynomial-time reduction Cook: Turing reduction Karp: Many-one reduction

6 Cook, Karp, Levin

Polynomial-time reduction Cook: Turing reduction Karp: Many-one reduction We use this for !p

6 Cook, Karp, Levin

Polynomial-time reduction Cook: Turing reduction Karp: Many-one reduction We use this for !p

Between NP languages

6 Cook, Karp, Levin

Polynomial-time reduction Cook: Turing reduction Karp: Many-one reduction We use this for !p

Between NP languages Levin: Karp + witnesses easily transformed back and forth

6 Cook, Karp, Levin

Polynomial-time reduction Cook: Turing reduction Karp: Many-one reduction We use this for !p

Between NP languages Levin: Karp + witnesses easily transformed back and forth Parsimonious: Karp + number of witnesses doesn’t change

6 NP-completeness

7 NP-completeness

A language L is NP-Hard if for all L’ in NP, L’ !p L

7 NP-completeness

A language L is NP-Hard if for all L’ in NP, L’ !p L

A language L is NP-Complete if it is NP-Hard and is in NP

7 NP-completeness

A language L is NP-Hard if for all L’ in NP, L’ !p L

A language L is NP-Complete if it is NP-Hard and is in NP

To efﬁciently solve all problems in NP, you need to efﬁciently solve L and nothing more

7 A simple NPC language

8 A simple NPC language

TMSAT = { (M,z,1n,1t) | ∃w, |w|

8 A simple NPC language

TMSAT = { (M,z,1n,1t) | ∃w, |w|

TMSAT is in NP: TMVAL = { (M,z,1n,1t,w) | |w|

8 A simple NPC language

TMSAT = { (M,z,1n,1t) | ∃w, |w|

TMSAT is in NP: TMVAL = { (M,z,1n,1t,w) | |w|

TMSAT is NP-hard: Given a language L in NP deﬁned as L = { x | ∃w, |w|

8 A simple NPC language

TMSAT = { (M,z,1n,1t) | ∃w, |w|

TMSAT is in NP: TMVAL = { (M,z,1n,1t,w) | |w|

TMSAT is NP-hard: Given a language L in NP deﬁned as L = { x | ∃w, |w|

Any “natural” NPC language?

8 Boolean Circuits

0 1

9 Boolean Circuits

0 1 Boolean valued wires, AND, OR, NOT, CONST gates, inputs, output, directed acyclic graph

9 Boolean Circuits

0 1 Boolean valued wires, AND, OR, NOT, CONST gates, inputs, output, directed acyclic graph

Circuit evaluation CKT-VAL: given (ckt,inputs) ﬁnd ckt’s boolean output value

9 Boolean Circuits

0 1 Boolean valued wires, AND, OR, NOT, CONST gates, inputs, output, directed acyclic graph

Circuit evaluation CKT-VAL: given (ckt,inputs) ﬁnd ckt’s boolean output value

Can be done very efﬁciently: CKT-VAL is in P

9 Boolean Circuits

0 1 Boolean valued wires, AND, OR, NOT, CONST gates, inputs, output, directed acyclic graph

Circuit evaluation CKT-VAL: given (ckt,inputs) ﬁnd ckt’s boolean output value

Can be done very efﬁciently: CKT-VAL is in P

CKT-SAT: given ckt, is there a “satisfying” input (output=1). In NP.

9 CKT-SAT is NP-Complete

10 CKT-SAT is NP-Complete

Reduce any NP language L to CKT-SAT

10 CKT-SAT is NP-Complete

Reduce any NP language L to CKT-SAT

Let’s start from the TM for verifying membership in L, with time bound T

10 CKT-SAT is NP-Complete

Reduce any NP language L to CKT-SAT

Let’s start from the TM for verifying membership in L, with time bound T

Build a circuit which on input w outputs what the TM outputs on (x,w), within T steps

10 CKT-SAT is NP-Complete

Reduce any NP language L to CKT-SAT

Let’s start from the TM for verifying membership in L, with time bound T

Build a circuit which on input w outputs what the TM outputs on (x,w), within T steps

This circuit is an instance of CKT-SAT

10 CKT-SAT is NP-Complete

Reduce any NP language L to CKT-SAT

Let’s start from the TM for verifying membership in L, with time bound T

Build a circuit which on input w outputs what the TM outputs on (x,w), within T steps

This circuit is an instance of CKT-SAT

Ensure reduction is poly-time

10 TM to Circuit (x,w)

11 TM to Circuit Wires for conﬁgurations: a bundle (x,w) for each tape cell, encoding (content,state), where state is encoded in the cell with the head

Circuitry for evolution: each bundle depends (as speciﬁed by the TM) on 3 neighboring bundles at the previous level

11 TM to Circuit Wires for conﬁgurations: a bundle (x,w) for each tape cell, encoding (content,state), where state is encoded in the cell with the head

Circuitry for evolution: each bundle depends (as speciﬁed by the TM) on 3 neighboring bundles at the previous level

(Part of) initial conﬁguration, namely w, to be plugged in as input

11 TM to Circuit x,q0 Wires for conﬁgurations: a bundle (x,w) for each tape cell, encoding (content,state), where state is encoded in the cell with the head

Circuitry for evolution: each bundle depends (as speciﬁed by the TM) on 3 neighboring bundles at the previous level

(Part of) initial conﬁguration, namely w, to be plugged in as input

11 TM to Circuit x,q0 w Wires for conﬁgurations: a bundle (x,w) for each tape cell, encoding (content,state), where state is encoded in the cell with the head

Circuitry for evolution: each bundle depends (as speciﬁed by the TM) on 3 neighboring bundles at the previous level

(Part of) initial conﬁguration, namely w, to be plugged in as input

11 TM to Circuit x,q0 w Wires for conﬁgurations: a bundle (x,w) for each tape cell, encoding (content,state), where state is encoded in the cell with the head

Circuitry for evolution: each bundle depends (as speciﬁed by the TM) on 3 neighboring bundles at the previous level

(Part of) initial conﬁguration, namely w, to be plugged in as input

T conﬁgurations, T bundles each

11 TM to Circuit x,q0 w Wires for conﬁgurations: a bundle (x,w) for each tape cell, encoding (content,state), where state is encoded in the cell with the head

Circuitry for evolution: each bundle depends (as speciﬁed by the TM) on 3 neighboring bundles at the previous level

(Part of) initial conﬁguration, namely w, to be plugged in as input

T conﬁgurations, T bundles each

Circuit size = O(T2)

11 TM to Circuit x,q0 w (x,w)

12 TM to Circuit x,q0 w (x,w) Reducing any NP language L to CKT-SAT

TM for verifying membership in L, time-bound T, and input x " A circuit which on input w outputs what the TM outputs on (x,w) within T steps

12 TM to Circuit x,q0 w (x,w) Reducing any NP language L to CKT-SAT

TM for verifying membership in L, time-bound T, and input x " A circuit which on input w outputs what the TM outputs on (x,w) within T steps

Poly-time reduction

12 TM to Circuit x,q0 w (x,w) Reducing any NP language L to CKT-SAT

TM for verifying membership in L, time-bound T, and input x " A circuit which on input w outputs what the TM outputs on (x,w) within T steps

Poly-time reduction

CKT-SAT is NP-complete

12 Other NP-complete problems

13 Other NP-complete problems

SAT and 3SAT

13 Other NP-complete problems

SAT and 3SAT

SAT: Are all given “clauses” simultaneously satisﬁable? (Conjunctive Normal Form)

13 Other NP-complete problems

SAT and 3SAT

SAT: Are all given “clauses” simultaneously satisﬁable? (Conjunctive Normal Form)

3SAT: Each clause has at most 3 literals

13 Other NP-complete problems

SAT and 3SAT

SAT: Are all given “clauses” simultaneously satisﬁable? (Conjunctive Normal Form)

3SAT: Each clause has at most 3 literals

CLIQUE, INDEP-SET, VERTEX-COVER

13 Other NP-complete problems

SAT and 3SAT

SAT: Are all given “clauses” simultaneously satisﬁable? (Conjunctive Normal Form)

3SAT: Each clause has at most 3 literals

CLIQUE, INDEP-SET, VERTEX-COVER

Hundreds (thousands?) more known

13 Other NP-complete problems

SAT and 3SAT

SAT: Are all given “clauses” simultaneously satisﬁable? (Conjunctive Normal Form)

3SAT: Each clause has at most 3 literals

CLIQUE, INDEP-SET, VERTEX-COVER

Hundreds (thousands?) more known

Shown using already known ones:

13 Other NP-complete problems

SAT and 3SAT

SAT: Are all given “clauses” simultaneously satisﬁable? (Conjunctive Normal Form)

3SAT: Each clause has at most 3 literals

CLIQUE, INDEP-SET, VERTEX-COVER

Hundreds (thousands?) more known

Shown using already known ones:

If L !p L1 and L1 !p L2, then L !p L2

13 CKT-SAT #p SAT

14 CKT-SAT #p SAT

Converting a circuit to a collection of clauses:

14 CKT-SAT #p SAT

Converting a circuit to a collection of clauses:

For each wire (connected component), add a variable

14 CKT-SAT #p SAT

Converting a circuit to a collection of clauses:

For each wire (connected component), add a variable

For each gate, add a clause involving variables for wires connected to the gate:

14 CKT-SAT #p SAT

Converting a circuit to a collection of clauses:

For each wire (connected component), add a variable

For each gate, add a clause involving variables for wires connected to the gate: x e.g. y AND z : (z⇒x), (z⇒y), (¬z ⇒ ¬x ∨ ¬y). i.e., (¬z ∨ x), (¬z ∨ y), (z ∨ ¬x ∨ ¬y).

14 CKT-SAT #p SAT

Converting a circuit to a collection of clauses:

For each wire (connected component), add a variable

For each gate, add a clause involving variables for wires connected to the gate: x e.g. y AND z : (z⇒x), (z⇒y), (¬z ⇒ ¬x ∨ ¬y). i.e., (¬z ∨ x), (¬z ∨ y), (z ∨ ¬x ∨ ¬y).

x and OR : (z x ∨ y), (¬z ¬x), (¬z ¬y). y z ⇒ ⇒ ⇒

14 SAT #p 3SAT

15 SAT #p 3SAT

Previous reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT !p 3SAT.

15 SAT #p 3SAT

Previous reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT !p 3SAT.

More directly:

15 SAT #p 3SAT

Previous reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT !p 3SAT.

More directly:

(a ∨ b ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ y), (¬y ∨ d ∨ e)

15 SAT #p 3SAT

Previous reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT !p 3SAT.

More directly:

(a ∨ b ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ y), (¬y ∨ d ∨ e)

Reduction needs 3SAT

15 SAT #p 3SAT

Previous reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT !p 3SAT.

More directly:

(a ∨ b ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ y), (¬y ∨ d ∨ e)

Reduction needs 3SAT

2SAT is in fact in P! [Exercise]

15 SAT #p 3SAT

Previous reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT !p 3SAT.

More directly:

(a ∨ b ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ d ∨ e) " (a ∨ b ∨ x), (¬x ∨ c ∨ y), (¬y ∨ d ∨ e)

Reduction needs 3SAT

2SAT is in fact in P! [Exercise]

Reduction not parsimonious (can you make it? [Exercise])

15 3SAT #p CLIQUE

16 3SAT #p CLIQUE

Clauses " Graph

16 3SAT #p CLIQUE

(x ∨ ¬y ∨ ¬z) Clauses " Graph

(w ∨ y)

(w ∨ x ∨ ¬z)

16 3SAT #p CLIQUE

(x ∨ ¬y ∨ ¬z) Clauses " Graph

vertices: each clause’s (w ∨ y) satisfying assignments (for its variables)

(w ∨ x ∨ ¬z)

16 3SAT #p CLIQUE

(x ∨ ¬y ∨ ¬z) Clauses " Graph

vertices: each clause’s (w ∨ y) satisfying assignments (for its variables) 1*0* 1*1* 0*1*

(w ∨ x ∨ ¬z)

16 3SAT #p CLIQUE

(x ∨ ¬y ∨ ¬z) Clauses " Graph

vertices: each clause’s (w ∨ y) satisfying assignments (for its variables) 1*0* 1*1* 0*1*

(w ∨ x ∨ ¬z)

16 3SAT #p CLIQUE

*000 (x ∨ ¬y ∨ ¬z) *111 *100 Clauses " Graph *101 *010 *110 *001 vertices: each clause’s (w ∨ y) satisfying assignments (for its variables) 1*0* 1*1* 0*1*

(w ∨ x ∨ ¬z)

16 3SAT #p CLIQUE

*000 (x ∨ ¬y ∨ ¬z) *111 *100 Clauses " Graph *101 *010 *110 *001 vertices: each clause’s (w ∨ y) satisfying assignments (for its variables) 1*0* 1*1* 0*1*

(w ∨ x ∨ ¬z)

16 3SAT #p CLIQUE

*000 (x ∨ ¬y ∨ ¬z) *111 *100 Clauses " Graph *101 *010 *110 *001 vertices: each clause’s (w ∨ y) satisfying assignments (for its variables) 1*0* 1*1* 0*1*

00*0 (w ∨ x ∨ ¬z) 11*1 10*0

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

01*1 01*0 10*1 11*0

16 3SAT #p CLIQUE

*000 (x ∨ ¬y ∨ ¬z) *111 *100 Clauses " Graph *101 *010 *110 *001 vertices: each clause’s (w ∨ y) satisfying assignments (for its variables) 1*0* 1*1* 0*1* edges between consistent assignments 00*0 (w ∨ x ∨ ¬z) 11*1 10*0 m-clique iff all m 01*1 clauses satisﬁable 01*0 10*1 11*0

16 3SAT #p CLIQUE

17 INDEP-SET and VERTEX-COVER

CLIQUE !p INDEP-SET

17 INDEP-SET and VERTEX-COVER

CLIQUE !p INDEP-SET

G has an m-clique iff G’ has an m-independent-set

17 INDEP-SET and VERTEX-COVER

CLIQUE !p INDEP-SET

G has an m-clique iff G’ has an m-independent-set

INDEP-SET !p VERTEX-COVER

17 INDEP-SET and VERTEX-COVER

CLIQUE !p INDEP-SET

G has an m-clique iff G’ has an m-independent-set

INDEP-SET !p VERTEX-COVER

G has an m-indep-set iff G has an (n-m)-vertex-cover

17 NP, P, co-NP and NPC

NP coNP

NPC coNPC

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

NP coNP

NPC coNPC

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

NP coNP

NPC coNPC

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

So is co-NP (If X is closed, so is co-X. Why?) NP coNP

NPC coNPC

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

So is co-NP (If X is closed, so is co-X. Why?) NP coNP If any NPC language is in P, then NP = P

NPC coNPC

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

So is co-NP (If X is closed, so is co-X. Why?) NP coNP If any NPC language is in P, then NP = P

NPC coNPC If any NPC language is in co-NP, then NP = co-NP P

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

So is co-NP (If X is closed, so is co-X. Why?) NP coNP If any NPC language is in P, then NP = P

NPC coNPC If any NPC language is in co-NP, then NP = co-NP P Note: X ⊆ co-X ⇒ X = co-X (Why?)

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

So is co-NP (If X is closed, so is co-X. Why?) NP coNP If any NPC language is in P, then NP = P

NPC coNPC If any NPC language is in co-NP, then NP = co-NP P Note: X ⊆ co-X ⇒ X = co-X (Why?)

L is NP-complete iff Lc is co-NP-complete (Why?)

18 NP, P, co-NP and NPC

We say class X is “closed under polynomial reductions” if (L1 !p L2 and L2 in class X) implies L1 in X

e.g. P, NP are closed under polynomial reductions

So is co-NP (If X is closed, so is co-X. Why?) NP coNP If any NPC language is in P, then NP = P

NPC coNPC If any NPC language is in co-NP, then NP = co-NP P Note: X ⊆ co-X ⇒ X = co-X (Why?)

L is NP-complete iff Lc is co-NP-complete (Why?)

co-NP complete = co-(NP-complete) 18 To d a y

19 To d a y

Polynomial-time reductions

19 To d a y

Polynomial-time reductions

NP-completeness (using Karp reductions)

19 To d a y

Polynomial-time reductions

NP-completeness (using Karp reductions)

Trivially, TMSAT

19 To d a y

Polynomial-time reductions

NP-completeness (using Karp reductions)

Trivially, TMSAT

Interestingly, CKT-SAT, SAT, 3SAT, CLIQUE, INDEP-SET, VERTEX-COVER

19 To d a y

Polynomial-time reductions

NP-completeness (using Karp reductions)

Trivially, TMSAT

Interestingly, CKT-SAT, SAT, 3SAT, CLIQUE, INDEP-SET, VERTEX-COVER

If any NPC language in P, then P=NP

19 Next Time

20 Next Time

Ladner’s Theorem: If NP ! P, then non-P, non-NPC languages

20 Next Time

Ladner’s Theorem: If NP ! P, then non-P, non-NPC languages

Time hierarchy theorems: More time, more power, strictly!