Intractability: P, NP, NP-Complete

�� Intractability

P,NP,NP-Complete 3-SAT

ET -S

AT poly-time reduces NDEPENDENT Tyler Moore 3-S to I

CSE 3353, SMU, Dallas, TX INDEPENDENT-SET DIR-HAM-CYCLE GRAPH-3-COLOR SUBSET-SUM Lecture 20

VERTEX-COVER HAM-CYCLE PLANAR-3-COLOR SCHEDULING

Some slides created by or adapted from Dr. Kevin Wayne. For more information see

http://www.cs.princeton.edu/~wayne/kleinberg-tardos. Some code reused from Python Algorithms by Magnus Lie SET-COVER TSP 3-SAT poly-time reduces to all of Hetland. these problems (and many, many more)

2 2 / 36

��

Decision problem. 8. INTRACTABILITY II �Problem � is a set of strings. �Instance ��is one string. ‣ P vs. NP �Algorithm � solves problem �: �� iff ��∈��. ‣ NP-complete Def. Algorithm � runs in polynomial time if for every string �, �� ‣ co-NP terminates in at most �� "steps", where ��⋅� is some polynomial. ‣ NP-hard length of s

Ex. Problem PRIMES��. �� Instance ��. �AKS algorithm PRIMES in ��steps�

3 3 / 36 4 / 36 ��

P. Decision problems for which there is a poly-time algorithm. Certification algorithm intuition. �Certifier views things from "managerial" viewpoint. �Certifier doesn't determine whether ��∈�� on its own; Problem Description Algorithm yes no rather, it checks a proposed proof � that ��∈��.

ULTIPLE M Is � a multiple of ��? grade-school division 51, 17 51, 16 Def. Algorithm �� is a certifier for problem � if for every string �, ��∈�� iff there exists a string � such that ��. REL-PRIME Are � and � relatively prime ? Euclid (300 BCE) 34, 39 34, 51

"certificate" or "witness" PRIMES Is � prime ? AKS (2002) 53 51 Def. �� is the set of problems for which there exists a poly-time certifier. Is the edit distance between dynamic niether acgggt EDIT-DISTANCE �� is a poly-time algorithm. � and � less than 5 ? programming neither ttttta � Certificate ��is of polynomial size: ��≤�� for some polynomial ��⋅� ⎡� � �⎤� ⎡� �⎤� ⎡� � �⎤� ⎡�⎤ � Gauss-Edmonds ⎢� ⎥� ⎢� ⎥� ⎢� ⎥� ⎢� ⎥ Is there a vector � that � � � �� L-SOLVE ⎢� − ⎥� ⎢� ⎥� ⎢� ⎥� ⎢� ⎥ satisfies ��? elimination ⎣⎢� � ��⎦⎥� ⎣⎢��⎦⎥� ⎣⎢� � �⎦⎥� ⎣⎢�⎦⎥

Is there a path between � depth-first search ST-CONN Remark. �� stands for nondeterministic polynomial time. �nd � in a graph ��? (Theseus)

5 6 5 / 36 6 / 36

��

COMPOSITES. Given an integer �, is � composite? 3-SAT. Given a CNF formula Φ, is there a satisfying assignment?

Certificate. A nontrivial factor � of �. Such a certificate exists iff � is Certificate. An assignment of truth values to the � boolean variables.

composite. Moreover ��≤��. Certifier. Check that each clause in Φ has at least one true literal.

Certifier. Check that �� and that � is a multiple of ��.

�� Φ�� ∨� � ∨� � ∧� � ∨� � ∨� � ∧� � ∨� � ∨� � �� or �� 437,669 = 541 × 809 ��

��

Conclusion. COMPOSITES ∈ ��. Conclusion. 3-SAT ∈ ��.

7 8 7 / 36 8 / 36 ��

HAM-PATH. Given an undirected graph ��, does there exist a simple NP. Decision problems for which there is a poly-time certifier. path � that visits every node?

Certificate. A permutation of the � nodes. Problem Description Algorithm yes no

⎡� � �⎤� ⎡� �⎤� ⎡� � �⎤� ⎡�⎤� Is there a vector � that Gauss-Edmonds ⎢� ⎥� ⎢� ⎥� ⎢� ⎥� ⎢� ⎥� L-SOLVE � �−� �� elimination ⎢� ⎥� ⎢� ⎥� ⎢� ⎥� ⎢� ⎥� Certifier. Check that the permutation contains each node in � exactly once, satisfies ��? ⎣⎢� � ��⎦⎥� ⎣⎢��⎦⎥� ⎣⎢� � �⎦⎥� ⎣⎢�⎦⎥� and that there is an edge between each pair of adjacent nodes. COMPOSITES Is � composite ? AKS (2002) 51 53

Does � have a nontrivial factor FACTOR ? (56159, 50) (55687, 50) less than ��?

Is there a truth assignment that �� ∨�� SAT ? ��∨�� satisfies the formula�? ��∨�� ∨��

3-COLOR Can the nodes of a graph � be ? colored with 3 colors?

�� Is there a simple path between HAM-PATH ? � and � that visits every node? Conclusion. HAM-PATH ∈ ��.

9 10 9 / 36 10 / 36

��

NP. Decision problems for which there is a poly-time certifier. P. Decision problems for which there is a poly-time algorithm. NP. Decision problems for which there is a poly-time certifier. EXP. Decision problems for which there is an exponential-time algorithm.

Claim. ��⊆��. �� Pf. Consider any problem � ∈ �. �� By definition, there exists a poly-time algorithm �� that solves �. �� Certificate ��ε, certifier ��. ▪

Claim. ��⊆��. Pf. Consider any problem � ∈ ��. �� By definition, there exists a poly-time certifier �� for �. �To solve input �, run �� on all strings � with ��≤��. �Return �� if �� returns �� for any of these potential certificates. ▪

Remark. Time-hierarchy theorem implies ��⊊��.

11 12 11 / 36 12 / 36 ��

Q. How to solve an instance of 3-SAT with � variables? Does P = NP? [Cook 1971, Edmonds, Levin, Yablonski, Gödel] A. Exhaustive search: try all �� truth assignments. Is the decision problem as easy as the certification problem?

Q. Can we do anything substantially more clever? Conjecture. No poly-time algorithm for 3-SAT. �� "intractable" � ��

��≠��

If yes. Efficient algorithms for 3-SAT, TSP, 3-COLOR, FACTOR, … If no. No efficient algorithms possible for 3-SAT, TSP, 3-COLOR, …

Consensus opinion. Probably no.

13 14 13 / 36 14 / 36

��

P ≠ NP. P ≠ NP.

��

15 16 15 / 36 16 / 36 ��

P = NP. � = ��, but only Ω�� algorithm for 3-SAT.

� ≠ ��, but with �� algorithm for 3-SAT.

�� = �� is independent (of ZFC axiomatic set theory). ��

17 18 17 / 36 18 / 36

��

Millennium prize. $1 million for resolution of �� problem. Some writers for the Simpsons and Futurama. �J. Steward Burns. ��. �David X. Cohen. ��. �Al Jean. ��. �Ken Keeler. ��. �Jeff Westbrook. ��.

Def. Problem � polynomial (Cook) reduces to problem � if arbitrary 8. INTRACTABILITY II instances of problem � can be solved using: �Polynomial number of standard computational steps, plus ‣ P vs. NP �Polynomial number of calls to oracle that solves problem �. ‣ NP-complete Def. Problem � polynomial (Karp) transforms to problem � if given any ‣ co-NP input � to �, we can construct an input � such that � is a �� instance of � ‣ NP-hard iff � is a �� instance of �. we require |y| to be of size polynomial in |x|

Note. Polynomial transformation is polynomial reduction with just one call to oracle for �, exactly at the end of the algorithm for �. Almost all previous �� reductions were of this form.

Open question. Are these two concepts the same with respect to ��?

we abuse notation ≤ p and blur distinction

24 21 / 36 22 / 36

��

NP-complete. A problem � ∈ �� with the property that for every CIRCUIT-SAT. Given a combinational circuit built from AND, OR, and NOT gates,

problem � ∈ ��, ��≤��. is there a way to set the circuit inputs so that the output is 1?

Theorem. Suppose � ∈ ��-complete. Then � ∈ � iff ��. Pf. ⇐ If ��, then � ∈ � because � ∈ ��. �� Pf. ⇒ Suppose � ∈ �. ∧ �Consider any problem � ∈ ��. Since ��≤��, we have � ∈ �. �This implies �� ⊆ �. �We already know � ⊆� ��. Thus ��. ▪ ¬ ∧

�� ∧ ∨ ∨

Fundamental question. Do there exist "natural" ��-complete problems? 1 0 ?? ?

��

25 26 23 / 36 24 / 36 ��

Theorem. CIRCUIT-SAT ∈ ��-complete. [Cook 1971, Levin 1973] Theorem. CIRCUIT-SAT ∈ ��-complete. Pf sketch. �Clearly, CIRCUIT-SAT ∈ ��. �Any algorithm that takes a fixed number of bits � as input and produces a �� or �� answer can be represented by such a circuit. �Moreover, if algorithm takes poly-time, then circuit is of poly-size.

sketchy part of proof; fixing the number of bits is important, and reflects basic distinction between algorithms and circuits

�Consider any problem � ∈ ��. It has a poly-time certifier ��: � ∈ � iff there exists a certificate � of length �� such that ��. �View �� as an algorithm with �� + �� input bits and convert it into a poly-size circuit �.

� first �� bits are hard-coded with �

� remaining �� bits represent (unknown) bits of � �Circuit � is satisfiable iff ��.

27 28 25 / 36 26 / 36

��

Ex. Construction below creates a circuit � whose inputs can be set so that it Remark. Once we establish first "natural" ��-complete problem, outputs � iff graph � has an independent set of size 2. others fall like dominoes.

�� ∧ Recipe. To prove that � ∈ ��-complete: �� ¬ �Step 1. Show that � ∈ ��. Step 2. Choose an ��-complete problem �. �� ∨ �Step 3. Prove that ��≤��. ∨ ∨ ��

∧ ∧ ∧ ∨ Theorem. If � ∈ ��-complete, � ∈ ��, and ��≤��, then � ∈ ��-complete. u Pf. Consider any problem � ∈ ��. Then, both ��≤�� and ��≤��. �By transitivity, ��≤��. ∧ ∧ ∧ Hence � ∈ ��-complete. ▪ by definition of by assumption v w � NP-complete

�� u-v u-w v-w u v w

1 0 1 ? ? ?

⎛�� ⎞� �� ⎜� ⎟� ⎝�� ⎠�� 29 30 27 / 36 28 / 36 ��

CIRCUIT-SAT CIRCUIT-SAT

ET -S 3-SAT 3-SAT

ET poly-time reduces -S AT NDEPENDENT 3-S to I AT poly-time reduces 3-S NDEPENDENT to I

INDEPENDENT-SET DIR-HAM-CYCLE GRAPH-3-COLOR SUBSET-SUM INDEPENDENT-SET DIR-HAM-CYCLE GRAPH-3-COLOR SUBSET-SUM

VERTEX-COVER HAM-CYCLE PLANAR-3-COLOR SCHEDULING VERTEX-COVER HAM-CYCLE PLANAR-3-COLOR SCHEDULING

CIRCUIT-SAT poly-time reduces to all of All of these problems (and many, many more) SET-COVER TSP these problems (and many, many more) SET-COVER TSP poly-time reduce to CIRCUIT-SAT.

35 36 29 / 36 30 / 36

��

Basic genres of NP-complete problems and paradigmatic examples. CIRCUIT-SAT �Packing + covering problems: SET-COVER, VERTEX-COVER,INDEPENDENT-SET. �Constraint satisfaction problems: CIRCUIT-SAT, SAT, 3-SAT. �Sequencing problems: HAM-CYCLE, TSP. Partitioning problems: 3D-MATCHING, 3-COLOR. 3-SAT �

ET Numerical problems: SUBSET-SUM, PARTITION. -S � AT poly-time reduces 3-S NDEPENDENT to I Practice. Most �� problems are known to be either in � or ��-complete.

INDEPENDENT-SET DIR-HAM-CYCLE GRAPH-3-COLOR SUBSET-SUM Notable exceptions. FACTOR, GRAPH-ISOMORPHISM, NASH-EQUILIBRIUM.

Theory. [Ladner 1975] Unless ��, there exist problems in �� that are VERTEX-COVER HAM-CYCLE PLANAR-3-COLOR SCHEDULING neither in � nor ��-complete.

All of these problems are NP-complete; they are SET-COVER TSP manifestations of the same really hard problem.

37 38 31 / 36 32 / 36 ��

Aerospace engineering. Optimal mesh partitioning for finite elements. Garey and Johnson. Computers and Intractability. Biology. Phylogeny reconstruction. Appendix includes over 300 ��-complete problems. � Chemical engineering. Heat exchanger network synthesis. Most cited reference in computer science literature. � Chemistry. Protein folding. Civil engineering. Equilibrium of urban traffic flow. Economics. Computation of arbitrage in financial markets with friction. Electrical engineering. VLSI layout. Environmental engineering. Optimal placement of contaminant sensors. Financial engineering. Minimum risk portfolio of given return. Game theory. Nash equilibrium that maximizes social welfare.

Mathematics. Given integer ��, …, ��, compute Mechanical engineering. Structure of turbulence in sheared flows. Medicine. Reconstructing 3d shape from biplane angiocardiogram. Operations research. Traveling salesperson problem. Physics. Partition function of 3d Ising model. Politics. Shapley-Shubik voting power. Recreation. Versions of Sudoko, Checkers, Minesweeper, Tetris. Statistics. Optimal experimental design. 39 40 33 / 36 34 / 36

��

Overwhelming consensus (still). P ≠ NP.

��

��≠��

Why we believe ��≠��.

��

42 43 35 / 36 36 / 36