Announcements

• Benchmarking for project 3 due tonight CSE 326: Data Structures • Last homework due on Friday at beginning of class NP Completeness • Final next Thursday – Scheduled for 8:30, might be moved to 10:30 Brian Curless – Final will be closed book/notes Spring 2008 – Up to and including MSTs

• Reading for this lecture: Weiss Chapter 9.7

2

Your First Task Try it with paper and pencil

Your company has to inspect a set of roads between cities by driving over each of them. Driving over the roads costs money (fuel), and there are a lot of roads. Your boss wants you to figure out how to drive over each road exactly once. You get a bonus if, after inspecting the last Which of these can you draw without lifting your road, the car is back where it started. pencil, drawing each line only once? Can you start and end at the same point? 3 4 Historical Puzzle: Seven Bridges of Königsberg Graph Problem for the Bridges of Königsberg

Pregel River

Island of Kneiphof

Want to cross bridges but… Want to cross all bridges but… can cross each bridge only once. Can cross each bridge only once 5 6

eiπ = −1 Euler Circuits and Tours Finding Euler Circuits Given a connected, undirected graph G = (V,), find an Euler circuit in G. • Euler tour: a path through a graph that visits each edge exactly once Euler Circuit Existence : • Euler circuit: an Euler tour that starts and ends at the Check to see that all vertices have even degree same vertex Running time = • Named after Leonhard Euler (1707-1783), who cracked this problem and founded graph theory in Euler Circuit Algorithm: 1. Do an edge walk from a start vertex until you 1736 are back at the start vertex. Mark each edge you visit, and do not revisit marked edges. You • Some observations for undirected graphs: never get stuck because of the even degree – An Euler circuit exists iff the graph is connected and each property. vertex has even degree (= # of edges on the vertex) 2. The walk is removed leaving several components each with the even degree – An Euler tour exists iff the graph is connected and either all property. Recursively find Euler circuits for vertices have even degree or exactly two have odd degree these. 3. Splice all these circuits into an Euler circuit

7 Running time = 8 Euler Circuit Example Euler Circuit Example

A A

B C B C

G G

D E D E

F F Euler(A) : Euler(A) : A B G E D G C A

9 10

Euler Circuit Example Euler Circuit Example

A A B C B C B C B C G G D E D E D E D E F F F F Euler(A) : Euler(B) Euler(A) : Euler(B): A B G E D G C A A B G E D G C A B D F E C B

11 12 Euler Circuit Example Euler Tour For an Euler Tour, exactly two vertices are odd, the rest A even. Using a similar algorithm, you can find a path between the two odd vertices. B C B C A G

D E D E B C D

F F Euler(A) : Euler(B): A B G E D G C A B D F E C B Splice E F G A B D F E C B G E D G C A 13 14

W. . Hamiltonian Circuits Hamilton Your Second Task (1805-1865) • Euler circuit: A cycle that goes through each edge exactly once Your boss is pleased…and assigns you a B C new task. • Hamiltonian circuit: A cycle that goes through each vertex exactly once G Your company has to send someone by car (except the first=last one) I to a set of cities. • Does graph I have: D E The primary cost is the exorbitant toll going – An Euler circuit? – A Hamiltonian circuit? into each city. B C • Does graph II have: Your boss wants you to figure out how to – An Euler circuit? G II drive to each city exactly once, returning in – A Hamiltonian circuit? the end to the city of origin. • Does the Hamiltonian circuit problem D E seem easier or harder? 15 16 Finding Hamiltonian Circuits Analysis of our Exhaustive Search Algorithm • Problem: Find a Hamiltonian circuit in a B C connected, undirected graph G. • Worst case Æ need to search G • One solution: Search through all paths to find all paths one that visits each vertex exactly once – How many paths? D E – Can use your favorite graph search algorithm • Can depict these paths as a (DFS!) to find various paths search tree • This is an exhaustive search (“brute force”) algorithm. B • Worst case Æ need to search all paths D G C – How many paths?? G E D E C G E

17 Etc. of paths from B 18 Search tree

Analysis of our Exhaustive Exponential Time Search Algorithm • Let the average branching factor of 1E+60 each node in this tree be B 1E+55 B 1E+50 • |V| vertices, each with ≈ B branches 1E+45 • Total number of paths ≈ B·B·B … ·B 2^N PC, since Big Bang D G C 1E+40 = 1.2^N 1E+35 N^5 1E+30 • Worst case Æ Exponential time! G E D E C G E N^3 1E+25 Etc. 5N 1E+20 PC, 1 day Search tree 1E+15 1E+10 of paths from B 100000 1 1 10 100 1000

19 20 Polynomial vs. Exponential Time The P

• The set P is defined as the set of all • Most of our far have been problems that can be solved in 2 O(log N), O(N), O(N log N) or O(N ) running polynomial worst case time time for inputs of size N – Also known as the polynomial time – These are all polynomial time algorithms complexity class – Their running time is O(Nk) for some k > 0 – All problems that have some algorithm • Exponential time BN is asymptotically worse whose running time is O(Nk) for some k than any polynomial function Nk for any k • Examples of problems in P: sorting, shortest path, Euler circuit, etc. 21 22

Problem Spaces Saving your job Simplified view: if a problem is not Try as you might, every solution you come polynomial-time solvable (not in P), then it is an exponential-time problem. with for the Hamiltonian Circuit problem runs in exponential time. Shorthand: EXPTIME – P solutions are fast You have to report back to your boss. – EXPTIME are slow • Sometimes viewed as “intractable” P

23 24 When is a problem easy? When is a problem hard?

• We’ve seen some “easy” graph problems: • Almost everything we’ve seen in class has had a near linear time algorithm – Graph search • But of course, computers can’t solve every – Shortest-path problem quickly. – Minimum Spanning Tree • In fact, there are perfectly reasonable sounding • Not easy for us to come up with, but easy problems that no computer could ever solve in any reasonable amount of time (as far as we for the computer, once we know algorithm. know!). – The Hamiltonian Circuit problem is one of these (as far as we know). More later on just how hard it is…

25 26

When is a problem hopeless? A Glimmer of Hope

• If the solution to a problem requires generating a result Suppose you have a problem that is at least that is exponential in size, then the algorithm must run in exponential time (even if just to print the solution!). decideable. (Many are!) • Some problems are “undecideable” – no algorithm can be given for solving them. – The : is it possible to specify any If the output can be checked for correctness algorithm, which, given an arbitrary program and input to that program, will always correctly determine in polynomial-time, then maybe a whether or not that program will enter an infinite loop? polynomial-time solution exists! – No! [Turing, 1936]

Alan Turing • We’ll focus on problems that have (1912-1954) a glimmer of hope… 27 28 The Complexity Class NP Why NP?

• Definition: NP is the set of all problems • NP stands for Nondeterministic Polynomial for which a given candidate solution can time be tested in polynomial time – Why “nondeterministic”? Corresponds to • Are the following in NP: algorithms that can guess a solution (if it – Hamiltonian circuit problem? exists) Æ the solution is then verified to be correct in polynomial time – Euler circuit problem? – Nondeterministic algorithms don’t exist – – All polynomial time algorithms? purely theoretical idea invented to understand how hard a problem could be

29 30

Your Chance to Win Problem Spaces (revisited) a Turing Award (and $$$) We can now augment our problem space to • It is generally believed that P ≠ NP, i.e. include NP as a superset of P. there are problems in NP that are not in P – But no one has been able to show even EXPTIME Whenever someone finds one such problem! a polynomial time solution – This is the fundamental open problem in NP to a problem currently theoretical computer science. believed to be in NP - P, – Nearly everyone has given up trying to it moves to P. P prove it. Instead, theoreticians prove theorems about what follows once we

31 assume P ≠ NP ! 32 P = NP? Your Third Task Perhaps instead P = NP, but that would seem to be even harder to prove… Your boss buys your story that others couldn’t solve the last problem. EXPTIME EXPTIME Again, your company has to send someone by car to a set of cities. The primary cost is distance traveled (which NP translates to fuel costs). Your boss wants you to figure out how to drive to P P each city exactly once, then returning to the first city, while staying within a fixed mileage budget C.

P ≠ NP P = NP 33 34

The Traveling Salesman Problem (TSP) Transforming HC into TSP

• This amounts to solving… • We can transform Hamiltonian Cycle into TSP. …The Traveling Salesman Problem: • Given graph G=(V, E): – Given a (fully connected) weighted graph G, – Assign weight of 1 to each edge and an integer C, – Augment the graph with edges until it is a complete – is there a cycle that visits all vertices with cost ≤ C? graph G’=(V, E’). • One of the canonical problems in computer – Assign weight of 2 to the new edges. science. – Let C = |V|. • Note difference from Hamiltonian cycle: graph is complete, and we care about weight.

35 36 Examples Polynomial-time transformation

• G’ has a TSP tour of weight |V| iff (if and only if) G has a Hamiltonian Cycle. – Proof: “obvious” B C • What was the cost of transforming HC into G TSP?

D E • In the end, because there is a polynomial- time transformation from HC to TSP, we say TSP is “at least as hard as” Hamiltonian cycle. 37 38

Satisfiability Satisfiability

In 1971, Stephen Cook studied the …and he proved something remarkable: Satisfiability Problem: – Given a Boolean formula (collections of Every problem in NP can be polynomially ANDs, ORs, NOTs) over a set of variables, transformed to the Satisfiability Problem. – is there a TRUE/FALSE assignment to the variables such that the Boolean formula Thus, Satisfiability is at least as hard as every evaluates to TRUE? other problem in NP, i.e., it is the hardest NP problem.

39 We call it an “NP-complete” problem. 40 Turing Machines The Steam Powered

The proof is intricate and depends on a In the mid-1980s, Alan Borning attributed a computing abstraction called a Turing peculiar implementation of a Turing Machine… Machine to Larry Ruzzo, giving rise to this mural:

which is generalized to allow guessing the answer.

41 42

NP-completeness What’s NP-complete

• In fact, Satisfiability can be polynomially reduced • Satisfiability of logic formulas to some other NP problems (and vice versa). • All sorts of constraint problems • These other problems are equivalent to • All sorts of graph problems, including: Satisfiability, and so all other problems in NP – Hamiltonian Circuits can be transformed to them, as well. – Traveling Salesman? • NP-complete problems thus form an – Graph coloring: decide if the vertices of a graph be colored using K colors, such that no two adjacent equivalence set in NP (all equivalently hard, i.e., vertices have the same color. the hardest). • Not an overstatement to say that every area of • Solving one would give a solution for all of them! computer science comes up against NP- – If any NP-complete problem is in P, then all of complete problems. NP is in P 43 44 One tweak and you could be in NP-complete Analyzing Your Hard Problem

• It’s amazing how “little” tweaks on a • Your problem seems really hard. problem change the complexity: • If you can transform an NP-complete – Euler circuit is in P, but Hamiltonian circuit is problem into the one you’ trying to solve, NP-complete. then you can stop working on your – Shortest path between two points is problem! computable in O(|V|2), but longest path is NP-complete. • …unless you really need that Turing award.

45 46

P, NP, NPC, and Exponential Coping with NP-Completeness Time Problems • All currently known 1. Settle for algorithms that are fast on average: algorithms for NP-complete EXPTIME Worst case still takes exponential time, but problems run in exponential worst case NPC doesn’t occur very often. time But some NP-Complete problems are also average- NP time NP-Complete! • Diagram depicts 2. Settle for fast algorithms that give near-optimal relationship between P, NP, solutions: In traveling salesman, may not give and EXPTIME (class of P the cheapest tour, but maybe good enough. problems that provably But finding even approximate solutions to some NP- require exponential time to Complete problems are NP-Complete! solve) It is believed that P ≠ NP ≠ EXPTIME

47 48 Coping with NP-Completeness Great Quick Reference 3. Just get the exponent as as possible! Is this lecture complete? Hardly, but here’s Much work on exponential algorithms for a good reference: satisfiability: in practice can often solve circuits with 1,000+ inputs But even 2n/100 will eventual hit the exponential curve! Computers and Intractability: 4. Restrict the problem: Longest Path is easy on A Guide to the Theory of trees, for example. Many hard problems are easy for restricted inputs. NP-Completeness by Michael S. Garey and David S. Johnson

49 50