American Scientist the Magazine of Sigma Xi, the Scientific Research Society
Total Page:16
File Type:pdf, Size:1020Kb
A reprint from American Scientist the magazine of Sigma Xi, The Scientific Research Society This reprint is provided for personal and noncommercial use. For any other use, please send a request Brian Hayes by electronic mail to [email protected]. Computing Science Accidental Algorithms Brian Hayes hy are some computational also refers to them as “accidental algo- Wproblems so hard and others A strange new family rithms,” emphasizing their capricious, easy? This may sound like a childish, rabbit-from-the-hat quality; they seem whining question, to be dismissed with to pluck answers from a tangle of un- a shrug or a wisecrack, but if you dress of algorithms probes likely coincidences and cancellations. it up in the fancy jargon of computation- I am reminded of the famous Sidney al complexity theory, it becomes quite the boundary Harris cartoon in which a long series a serious and grownup question: Is P of equations on a blackboard hinges on equal to NP? An answer—accompanied between easy the notation “Then a miracle occurs.” by a proof—will get you a million bucks from the Clay Mathematics Institute. and hard problems The Coffee-Break Criterion I’ll return in a moment to P and NP, For most of us, the boundary between but first an example, which offers a fast and slow computations is clearly glimpse of the mystery lurking beneath exactly once? This problem was posed marked: A computation is slow if it’s the surface of hard and easy problems. in 1858 by William Rowan Hamilton, not finished when you come back Consider a mathematical graph, a col- and the path is called a Hamiltonian from a coffee break. Computer science lection of vertices (represented by dots) circuit. Again we can get the answer formalizes this definition in terms of and edges (lines that connect the dots). by brute force. But in this case there is polynomial-time and exponential-time Here’s a nicely symmetrical example: no trick like Euler’s; no one knows any algorithms. method that gives the correct answer Suppose you are running a comput- for all graphs and does so substantially er program whose input is a list of n quicker than exhaustive search. Super- numbers. The program might be sort- ficially, the two problems look almost ing the numbers, or finding their great- identical, but Hamilton’s version is far est common divisor, or generating per- harder. Why? Is it because no shortcut mutations of them. No matter what the Is it possible to construct a path that solution exists, or have we not yet been task, the running time of the program traverses each edge exactly once and clever enough to find one? will likely depend in some way on n, returns to the starting point? For any Most computer scientists and math- the length of the list (or, more precisely, graph with a finite number of edges, ematicians believe that Hamilton’s on the total number of bits needed to we could answer such a question by problem really is harder, and no short- represent the numbers). Perhaps the brute force: Simply list all possible cut algorithm will ever be found—but time needed to process n items grows paths and check to see whether any of that’s just a conjecture, supported by as n2. Thus as n increases from 10 to 20 them meet the stated conditions. But experience and intuition but not by to 30, the running time rises from 100 there’s a better way. In 1736 Leonhard proof. Contrarians argue that we’ve to 400 to 00. Now consider a program Euler proved that the desired path hardly begun to explore the space of whose running time is equal to 2n. In (now called an Eulerian circuit) exists all possible algorithms, and new prob- this case, as the size of the input grows if and only if every vertex is the end lem-solving techniques could turn up from 10 to 20 to 30, the running time point of an even number of edges. We at any time. Before 1736, the Eulerian- leaps from a thousand to a million to can check whether a graph has this circuit problem also looked hard. a billion. You’re going to be drinking a property without any laborious enu- What prompts me to write on this lot of coffee. meration of pathways. theme is a new and wholly unexpected The function n2 is an example of a Now take the same graph and ask family of algorithms that provide ef- polynomial; 2n denotes an exponential. a slightly different question: Is there a ficient methods for several problems The distinction between these catego- circuit that passes through every vertex that previously had only brute-force ries of functions marks the great divide solutions. The algorithms were invent- of computational complexity theory. Brian Hayes is Senior Writer for American Sci- ed by Leslie G. Valiant of Harvard Uni- Roughly speaking, polynomial algo- entist. Additional material related to the “Comput- versity, with extensive further contri- rithms are fast and efficient; exponen- ing Science” column appears in Hayes’s Weblog at butions by Jin-Yi Cai of the University tial algorithms are too slow to bother http://bit-player.org. Address: 211 Dacian Avenue, of Wisconsin. Valiant named the meth- with. To speak a little less roughly: Durham, NC 27701. Internet: [email protected] ods “holographic algorithms,” but he When n becomes large enough, any www.americanscientist.org © 2008 Brian Hayes. Reproduction with permission only. 2008 January–February Contact [email protected]. one polynomial-time algorithm. The pute an answer in polynomial time, but algorithm has to give the right answer if you happen to guess the answer, or if and has to run in polynomial time on someone whispers it in your ear, then O every instance of the problem. you can quickly verify its correctness. Classifying problems for which we NP is the complexity class for conscien- don’t know a polynomial-time algo- tious cheaters—students who don’t do rithm is where it gets tricky. In the first their own homework but who at least place, there are some problems that check their cribbed answers before they G O require exponential running time for turn them in. O reasons that aren’t very interesting. Detecting a Hamiltonian circuit is Think about a program to generate all one example of a problem in NP. Even subsets of a set of n items; the compu- though I don’t know how to solve the n tation is easy, but because there are 2 problem efficiently for all graphs, if subsets, just writing down the answer you show me a purported Hamiltoni- O will take an exponential amount of an circuit, I can readily check whether time. To avoid such issues, complex- it passes through every vertex once: ity theory focuses on problems with O short answers. Decision problems ask Polynomial and exponential functions define a yes-or-no question (“Does the graph the poles of computational efficiency. The have a Hamiltonian circuit?”). There running time of an algorithm is measured as are also counting problems (“How a function of the size of the input, n. If the many Hamiltonian circuits does the function is a polynomial one, such as n or graph have?”). Problems of these (Note that this verification scheme n2, the algorithm is considered efficient; an n kinds might conceivably have a poly- works only when the answer to the exponential growth rate, such as 2 , makes nomial-time solution, and we know decision problem is “yes.” If you claim the algorithm impractically slow. that some of them do. The big question that a graph doesn’t have a Hamilto- is whether all of them do. If not, what nian circuit, the only way to prove it is polynomial-time program is faster distinguishes the easy problems from to enumerate all possible paths.) than any exponential-time program. the hard ones? Within the class NP dwells the So much for the classification of al- elite group of problems labeled NP- gorithms. What about classifying the Conscientious Cheating complete. They have an extraordinary problems that the algorithms are sup- The letters NP might well be translated property: If any one of these problems posed to solve? For any given prob- “notorious problems,” but the abbrevi- has a polynomial-time solution, then lem, there might be many different al- ation actually stands for “nondetermin- that method can be adapted to quick- gorithms, some faster than others. The istic polynomial.” The term refers to a ly solve all problems in NP (both the custom is to rate a problem according hypothetical computing machine that complete ones and the rest). In other to the worst-case performance of the can solve problems through systematic words, such an algorithm would es- best algorithm. The class known as P guesswork. For the problems in NP, tablish that P = NP. The two categories includes all problems that have at least you may or may not be able to com- would merge. The very concept of NP-complete- ness has a whiff of the miraculous about it. How can you possibly be sure that a solution to one problem will work for every other problem in NP as well? After all, you can’t even know in advance what all those problems are. The answer is so curious and improb- able that it’s worth a brief digression. The first proof of NP-completeness, published in 171 by Stephen A. Cook of the University of Toronto, concerns a problem called satisfiability.