Lecture 27: The Grand Finale The“complexity”classNP

✦ Agenda for the final class: ✦ Definition: NP is the set of all problems for which a given ➭ P, NP, and NP-completeness candidate solution can be tested in polynomial time ➭ The NP =? P problem ➭ Suppose someone gives you a solution – can it be tested in ➧ Major extra-credit problem polynomial time? (testing is easier than solving it) ➭ Final Review ➧ Summary of what you’ve learned in this course ✦ Example of a problem in NP: ➭ End-of-quarter M.o.M. party ➭ Our new friend, the Hamiltonian circuit problem:Whyis ➧ Munchonmunchiesasyouleave… it in NP? ➧ Given a candidate path, can test in linear time if it is a Hamiltonian circuit – just check if all vertices are visited exactly once in the candidate path (except start/finish vertex)

R. Rao, CSE 373 1 R. Rao, CSE 373 3

From Last Time: The “complexity” class P Why NP?

✦ The set P is defined as the set of all problems that can be ✦ NP stands for Nondeterministic Polynomial time solved in polynomial worse case time ➭ Why “nondeterministic”? Corresponds to algorithms that can ➭ Also known as the polynomial time complexity class – search all possible solutions in parallel and pick the correct contains problems whose time complexity is O(Nk)forsomek one ! each solution can be checked in polynomial time ➭ Nondeterministic algorithms don’t exist – purely theoretical ✦ Examples of problems in P: searching, sorting, topological idea invented to understand how hard a problem could be sort, single-source shortest path, Euler circuit, etc. ✦ Examples of problems in NP: ➭ Hamiltonian circuit: Given a candidate path, can test in linear time if it is a Hamiltonian circuit ➭ Sorting: Can test in linear time if a candidate ordering is sorted ➭ Sorting is also in P. Are any other problems in P also in NP?

R. Rao, CSE 373 2 R. Rao, CSE 373 4 More revelations about NP NP-complete problems

✦ Sorting is in P. Are any other problems in P also in NP? ✦ The “hardest” problems in NP are called NP-complete ➭ YES! All problems in P are also in NP ! P ⊆ NP (NPC) problems ➧ If you can solve a problem in polynomial time, can ✦ Why “hardest”? A problem X is NP-complete if: definitely verify a solution in polynomial time 1. XisinNPand 2. any converted to ✦ Question: Are all problems in NP also in P? problem Y in NP can be Xin ⊆ polynomial time, such that solving X also provides a ➭ Is NP P? solution for Y ! Can use algorithm for X as a subroutine to solve Y ✦ Thus, if you find a poly time algorithm for just one NPC problem, all problems in NP can be solved in poly time ✦ Example: The Hamiltonian circuit problem can be shown to be NP-complete (not so easy to prove!) R. Rao, CSE 373 5 R. Rao, CSE 373 7

Your chance to win a Turing award: P = NP? P, NP, and Exponential Time Problems

✦ Nobody knows whether NP ⊆ P ✦ All algorithms for NP- ➭ Proving or disproving this will bring you instant fame! complete problems so far have EXPTIME tended to run in nearly ✦ It is generally believed that P ≠ NP i.e. there are problems in NPC NP that are not in P exponential worst case time ➭ But this doesn’t mean fast ➭ But no one has been able to show even one such problem NP sub-exponential time ✦ A very large number of problems are in NP (such as the algorithms don’t exist! Not Hamiltonian circuit problem) proven yet… P ➭ No one has found fast (polynomial time) algorithms for these ✦ Diagram depicts relationship problems betweenP,NP,andEXPTIME ➭ On the other hand, no one has been able to prove such algorithms don’t exist (i.e. that these problems are not in P)! (class of problems that require It is believed that exponential time to solve) P ≠ NP ≠ EXPTIME

R. Rao, CSE 373 6 R. Rao, CSE 373 8 The “graph” of NP-completeness TSP is NP-complete!

✦ Cook first showed (in 1971) ✦ We can show TSP is also NPC if we can convert any input that satisfiability of Boolean forHCtoaninputforTSPinpolytime.Here’soneway: formulas (SAT) is NP-complete ✦ Hundreds of other problems 1 B C B C (from scheduling and databases 1 1 to optimization theory) have HC G TSP since been shown to be NPC G 111 1 ✦ How? By showing an algorithm D E 2 D 1 E 2 that converts a known NPC problem to your pet problem in poly time ! then, your problem is also NPC! This graph has a Hamiltonian circuit iff this fully-connected graph has a TSP cycle of total cost ≤ K, where K = |V|(here,K=5)

R. Rao, CSE 373 9 R. Rao, CSE 373 11

Showing NP-completeness: An example Coping with NP-completeness 4 ✦ Consider the Traveling B C ✦ Given that it is difficult to find fast algorithms for NPC Salesperson (TSP) Problem: 1 Cycle problems, what do we do? Given a fully connected, weighted 3 2 with cost graph G = (V,E), is there a cycle 1 ≤ 8 ! ✦ Alternatives: that visits all vertices exactly once 3 BDCEB 1. Dynamic programming: Avoid repeatedly solving the same and has total cost ≤ K? D E subproblem – use table to store results (see Chap. 10) 2. Settle for algorithms that are fast on average: Worst case still ✦ TSPisinNP(why?) B C takes exponential time, but doesn’t occur very often ✦ Can we show TSP is NP- Convert 3. Settle for fast algorithms that give near-optimal solutions:In complete? G TSP, may not give the cheapest tour, but maybe good enough ➭ Hamiltonian Circuit (HC) is NPC to input ➭ Can show TSP is also NPC if we for TSP 4. Try to get a “wimpy exponential” time algorithm: It’s okay if can convert any input for HC to running time is O(1.00001N) – bad only for N > 1,000,000 an input for TSP in poly time D E R. Rao, CSE 373 Input for HC 10 R. Rao, CSE 373 12 Yawn…What does all this have to do with data structures and programming? Final Review: What you need to know ✦ Top 5 reasons to know and understand NP-completeness: ✦ Basic Math N N(N +1) ∑i = ➭ Logs, exponents, summation of series i=1 2 5. What if there’s an NP-completeness question in the final? + ➭ Proof by induction N AN 1 −1 ∑ Ai = = A − 4. When you are having a tough time programming a fast ✦ Asymptotic Analysis i 0 1 algorithm for a problem, you could show it is NP-complete ➭ Big-oh, little-oh, Theta and Omega 3. When you are having a tough time programming a fast ➭ Know the definitions and how to show f(N) is big-oh/little- oh/Theta/Omega of (g(N)) algorithm for a problem, you could just say it is NPC (and ➭ How to estimate Running Time of code fragments many will believe you (yes, it’s a sad state of affairs)) ➧ E.g. nested “for” loops 2. When you are at a cocktail party, you can impress your ✦ Recurrence Relations friends with your profound knowledge of NP-completeness ➭ Deriving recurrence relation for run time of a recursive 1. Make money with new T-shirt slogan: “And God said: P=NP” function ➭ Solving recurrence relations by expansion to get run time

R. Rao, CSE 373 13 R. Rao, CSE 373 15

Final Review: What you need to know

✦ Lists, Stacks, Queues ➭ Brush up on ADT operations – Insert/Delete, Push/Pop etc. ➭ Array versus pointer implementations of each data structure Final Review ➭ Header nodes, circular, doubly linked lists ✦ Trees ➭ Definitions/Terminology: root, parent, child, height, depth etc. (“We’ve covered way too much in this course… ➭ Relationship between depth and size of tree ➧ What do I really need to know?”) Depth can be between O(log N) and O(N) for N nodes

R. Rao, CSE 373 14 R. Rao, CSE 373 16 Final Review: What you need to know Final Review: What you need to know

✦ Binary Search Trees ✦ Sorting Algorithms: Know run times and how they work ➭ How to do Find, Insert, Delete ➭ Elementary sorting algorithms and their run time ➧ Bad worst case performance – could take up to O(N) time ➧ Bubble sort, Selection sort, Insertion sort ➭ AVL trees ➧ ➭ Shellsort – based on several passes of Insertion sort Balance factor is +1, 0, -1 ➧ ➧ Know single and double rotations to keep tree balanced Increment Sequence ➧ All operations are O(log N) worst case time ➭ Heapsort – based on binary heaps (max-heaps) ➭ Splay trees – good amortized performance ➧ BuildHeap and repeated DeleteMax’s ➧ AsingleoperationmaytakeO(N)timebutinasequenceof ➭ Mergesort – recursive divide-and-conquer, uses extra array operations, average time per operation is O(log N) ➭ Quicksort – recursive divide-and-conquer, Partition in-place ➧ Every Find, Insert, Delete causes accessed node to be ➧ fastest in practice, but O(N2) worst case time moved to the root ➧ Pivot selection – median-of-three works best ➧ Know how to zig-zig, zig-zag, etc. to “bubble” node to top ➭ Know which of these are stable and in-place ➭ B-trees: Know basic idea behind Insert/Delete ➭ Lower bound on sorting, bucket sort, and radix sort

R. Rao, CSE 373 17 R. Rao, CSE 373 19

Final Review: What you need to know Final Review: What you need to know

✦ Priority Queues ✦ Disjoint Sets and Union-Find ➭ Binary Heaps: Insert/DeleteMin, Percolate up/down ➭ Up-trees and their array-based implementation ➧ Array implementation ➭ Know how Union-by-size and Path compression work ➧ BuildHeap takes only O(N) time (used in heapsort) ➭ No need to know run time analysis – just know the result: ➭ Binomial Queues: Forest of binomial trees with heap order ➧ Sequence of M operations with Union-by-size and P.C. is ➧ Merge is fast – O(log N) time Θ(M α(M,N)) – basically Θ(1) amortized time per op ➧ Insert and DeleteMin based on Merge ✦ Graph Algorithms ✦ Hashing ➭ Adjacency matrix versus adjacency list representation of ➭ Hash functions based on the mod function graphs ➭ Collision resolution strategies ➭ Know how to Topological sort in O(|V|+|E|) time using a ➧ Chaining, Linear and Quadratic probing, Double Hashing queue ➭ Load factor of a hash table ➭ Breadth First Search (BFS) for unweighted shortest path

R. Rao, CSE 373 18 R. Rao, CSE 373 20 Final Review: What you need to know

✦ Graph Algorithms (cont.) ➭ Dijkstra’s shortest path algorithm – greed works! ➧ Know how a priority queue can speed up the algorithm ➭ Depth First Search (DFS) ➭ Minimum Spanning trees: Know the 2 greedy algorithms ➧ Prim’s algorithm – similar to Dijkstra’s algorithm ➧ Kruskal’s algorithm Know how it uses a priority queue and Union/Find ➧ Euler versus Hamiltonian circuits – difference in run times ➧ Know what P, NP, and NP-completeness mean How one problem can be “reduced” to another (e.g. input to HC can be transformed into input for TSP)

R. Rao, CSE 373 21

Final Exam: Where: This room When: Wednesday, June 6, 2:30-4:20pm To Do: Go over sample final exam on web site Prepare, prepare, prepare (for the final) Have a great summer!

R. Rao, CSE 373 22