A reprint from American Scientist the magazine of Sigma Xi, The Scientific Research Society

This reprint is provided for personal and noncommercial use. For any other use, please send a request Brian Hayes by electronic mail to [email protected]. Computing Science

Accidental Algorithms

Brian Hayes

hy are some computational also refers to them as “accidental algo- Wproblems so hard and others A strange new family rithms,” emphasizing their capricious, easy? This may sound like a childish, rabbit-from-the-hat quality; they seem whining question, to be dismissed with to pluck answers from a tangle of un- a shrug or a wisecrack, but if you dress of algorithms probes likely coincidences and cancellations. it up in the fancy jargon of computation- I am reminded of the famous Sidney al complexity theory, it becomes quite the boundary Harris cartoon in which a long series a serious and grownup question: Is P of equations on a blackboard hinges on equal to NP? An answer—­accompanied between easy the notation “Then a miracle occurs.” by a proof—­will get you a million bucks from the Clay Mathematics Institute. and hard problems The Coffee-Break Criterion I’ll return in a moment to P and NP, For most of us, the boundary between but first an example, which offers a fast and slow computations is clearly glimpse of the mystery lurking beneath exactly once? This problem was posed marked: A computation is slow if it’s the surface of hard and easy problems. in 1858 by William Rowan Hamilton, not finished when you come back Consider a mathematical graph, a col- and the path is called a Hamiltonian from a coffee break. Computer science lection of vertices (represented by dots) circuit. Again we can get the answer formalizes this definition in terms of and edges (lines that connect the dots). by brute force. But in this case there is polynomial-time and exponential-time Here’s a nicely symmetrical example: no trick like Euler’s; no one knows any algorithms. method that gives the correct answer Suppose you are running a comput- for all graphs and does so substantially er program whose input is a list of n quicker than exhaustive search. Super- numbers. The program might be sort- ficially, the two problems look almost ing the numbers, or finding their great- identical, but Hamilton’s version is far est common divisor, or generating per- harder. Why? Is it because no shortcut mutations of them. No matter what the Is it possible to construct a path that solution exists, or have we not yet been task, the running time of the program traverses each edge exactly once and clever enough to find one? will likely depend in some way on n, returns to the starting point? For any Most computer scientists and math- the length of the list (or, more precisely, graph with a finite number of edges, ematicians believe that Hamilton’s on the total number of bits needed to we could answer such a question by problem really is harder, and no short- represent the numbers). Perhaps the brute force: Simply list all possible cut algorithm will ever be found—­but time needed to process n items grows paths and check to see whether any of that’s just a conjecture, supported by as n2. Thus as n increases from 10 to 20 them meet the stated conditions. But experience and intuition but not by to 30, the running time rises from 100 there’s a better way. In 1736 Leonhard proof. Contrarians argue that we’ve to 400 to 900. Now consider a program Euler proved that the desired path hardly begun to explore the space of whose running time is equal to 2n. In (now called an Eulerian circuit) exists all possible algorithms, and new prob- this case, as the size of the input grows if and only if every vertex is the end lem-solving techniques could turn up from 10 to 20 to 30, the running time point of an even number of edges. We at any time. Before 1736, the Eulerian- leaps from a thousand to a million to can check whether a graph has this circuit problem also looked hard. a billion. You’re going to be drinking a property without any laborious enu- What prompts me to write on this lot of coffee. meration of pathways. theme is a new and wholly unexpected The function n2 is an example of a Now take the same graph and ask family of algorithms that provide ef- polynomial; 2n denotes an exponential. a slightly different question: Is there a ficient methods for several problems The distinction between these catego- circuit that passes through every vertex that previously had only brute-force ries of functions marks the great divide solutions. The algorithms were invent- of computational complexity theory. Brian Hayes is Senior Writer for American Sci- ed by Leslie G. Valiant of Harvard Uni- Roughly speaking, polynomial algo- entist. Additional material related to the “Comput- versity, with extensive further contri- rithms are fast and efficient; exponen- ing Science” column appears in Hayes’s Weblog at butions by Jin-Yi Cai of the University tial algorithms are too slow to bother http://bit-player.org. Address: 211 Dacian Avenue, of Wisconsin. Valiant named the meth- with. To speak a little less roughly: Durham, NC 27701. Internet: [email protected] ods “holographic algorithms,” but he When n becomes large enough, any www.americanscientist.org © 2008 Brian Hayes. Reproduction with permission only. 2008 January–February  Contact [email protected].  one polynomial-time algorithm. The pute an answer in polynomial time, but algorithm has to give the right answer if you happen to guess the answer, or if and has to run in polynomial time on someone whispers it in your ear, then O  every instance of the problem. you can quickly verify its correctness. Classifying problems for which we NP is the complexity class for conscien- don’t know a polynomial-time algo- tious cheaters—­students who don’t do  rithm is where it gets tricky. In the first their own homework but who at least place, there are some problems that check their cribbed answers before they

G O require exponential running time for turn them in. O  reasons that aren’t very interesting. Detecting a Hamiltonian circuit is Think about a program to generate all one example of a problem in NP. Even subsets of a set of n items; the compu- though I don’t know how to solve the n  tation is easy, but because there are 2 problem efficiently for all graphs, if subsets, just writing down the answer you show me a purported Hamiltoni- O will take an exponential amount of an circuit, I can readily check whether time. To avoid such issues, complex- it passes through every vertex once:       ity theory focuses on problems with O short answers. Decision problems ask Polynomial and exponential functions define a yes-or-no question (“Does the graph the poles of computational efficiency. The have a Hamiltonian circuit?”). There running time of an algorithm is measured as are also counting problems (“How a function of the size of the input, n. If the many Hamiltonian circuits does the function is a polynomial one, such as n or graph have?”). Problems of these (Note that this verification scheme n2, the algorithm is considered efficient; an n kinds might conceivably have a poly- works only when the answer to the exponential growth rate, such as 2 , makes nomial-time solution, and we know decision problem is “yes.” If you claim the algorithm impractically slow. that some of them do. The big question that a graph doesn’t have a Hamilto- is whether all of them do. If not, what nian circuit, the only way to prove it is polynomial-time program is faster distinguishes the easy problems from to enumerate all possible paths.) than any exponential-time program. the hard ones? Within the class NP dwells the So much for the classification of al- elite group of problems labeled NP- gorithms. What about classifying the Conscientious Cheating ­complete. They have an extraordinary problems that the algorithms are sup- The letters NP might well be translated property: If any one of these problems posed to solve? For any given prob- “notorious problems,” but the abbrevi- has a polynomial-time solution, then lem, there might be many different al- ation actually stands for “nondetermin- that method can be adapted to quick- gorithms, some faster than others. The istic polynomial.” The term refers to a ly solve all problems in NP (both the custom is to rate a problem according hypothetical computing machine that complete ones and the rest). In other to the worst-case performance of the can solve problems through systematic words, such an algorithm would es- best algorithm. The class known as P guesswork. For the problems in NP, tablish that P = NP. The two categories includes all problems that have at least you may or may not be able to com- would merge. The very concept of NP-complete- ness has a whiff of the miraculous about it. How can you possibly be sure that a solution to one problem will work for every other problem in NP as well? After all, you can’t even know in advance what all those problems are. The answer is so curious and improb- able that it’s worth a brief digression. The first proof of NP-completeness, published in 1971 by Stephen A. Cook of the University of Toronto, concerns a problem called satisfiability. You are given a formula in Boolean logic, con- structed from a set of variables, each of which can take on the values true or false, and the logical connectives and, QMBOBS
HSJE QMBOBS OPOQMBOBS or and not. The decision problem asks: Is there a way of assigning true The perfect-matching problem pairs up the vertices of a mathematical graph. The number of possible matchings grows exponentially with the size of the graph; nevertheless, the match- and false values to the variables that ings can be counted in polynomial time on a planar graph (one without crossed edges). The makes the entire formula true? With n n problem was first studied on graphs with a periodic structure, such as the rectilinear grid at variables there are 2 possible assign- left, but the algorithm also works on less-regular planar graphs, such as the one in the middle. ments, so the brute-force approach is The graph at right is nonplanar, and its perfect matchings cannot be counted quickly. exponential and unappealing. But a

10 American Scientist, Volume 96 © 2008 Brian Hayes. Reproduction with permission only. Contact [email protected]. lucky guess is easily verified, so the which means the decision problem for problem qualifies as a member of NP. perfect matching is in P. (Indeed, Ed- HSBQI TLFX
BEKBDFODZ
NBUSJY Cook’s proof of NP-completeness is monds’s 1965 paper includes the first B E B C D E beautiful in its conception and a sham- published discussion of the distinction B     bling Rube Goldberg contraption in its between polynomial and exponential C m    details. The key insight is that Bool- algorithms.) C D D ean formulas can describe the circuits Another success story among match-  m   and operations of a computer. Cook ing methods applies only to planar E m  m  showed how to write an intricate for- graphs—­those that can be drawn with- mula that encodes the entire operation out crossed edges. On a planar graph 1G

BC
¤
DE
m
BD
¤
CE

BE
¤
CD

 of a computer executing a program you can efficiently solve not only the to guess a solution and check its cor- decision problem for perfect matching rectness. If and only if the Boolean but also the counting problem—­that is, formula has a satisfying assignment, you can learn how many different sub- the simulated computer program suc- sets of edges yield a perfect matching. ceeds. Thus if you could determine in In general, counting problems seem QFSGFDU
NBUDIJOHT polynomial time whether or not any more difficult than decision problems, Boolean formula is satisfiable, you since the solution conveys more infor- The fast algorithm for counting planar per- could also solve the encoded decision mation. The main complexity class for fect matchings works by translating the problem. The proof doesn’t depend counting problems is called #P (pro- problem into the language of matrices and on the details of that problem, only on nounced “sharp P”); it includes NP linear algebra. The pattern of connections the fact that it has a polynomial-time as a subset, so #P problems must be at within the graph is encoded in an adjacency checking procedure. least as hard as NP. matrix—an array of numbers with rows and columns labeled by the vertices of the graph. Thousands of problems are now The problem of counting planar per- If two vertices are joined by an edge, the cor- known to be NP-complete. They form fect matchings has its roots in phys- responding element of the matrix is either +1 a vast fabric of interdependent compu- ics and chemistry, where the original or –1; otherwise the element is 0. Elements of tations. Either all of them are hard, or question was: If diatomic molecules the matrix are combined in a sum of products everything in NP is easy. are adsorbed on a surface, forming a called the Pfaffian, which yields the number single layer, how many ways can they of perfect matchings. For the simple graph The Match Game be arranged? Another version asks shown here there are two perfect matchings. To understand the new holographic how many ways dominos (2-by-1 rect- algorithms, we need one more ingredi- angles) can be placed on a chessboard n! is not a polynomial function of n; it ent from graph theory: the idea of a without gaps or overlaps. The answers qualifies as an exponential. Thus, un- perfect matching. exhibit clear signs of exponential der the rules of complexity theory, the Consider the double-feature festi- growth; when you arrange dominos whole scheme is really no better than val. You want to show movies in pairs, on square boards of size 2, 4, 6 and 8, the brute-force enumeration of all per- with the proviso that any two films the number of distinct tilings is 2, 36, fect matchings. But this is where the scheduled together should have a per- 6,728 and 12,988,816. Given this rapid rabbit comes out of the hat. There are former in common; also, no film can proliferation, it seems quite remark- alternative algorithms for computing be screened more than once. These able that a polynomial-time algorithm determinants that do achieve polyno- constraints lead to a graph where the can count the configurations. The in- mial performance; the best-known ex- vertices are film titles, and two titles genious method was developed in the ample is the technique called Gaussian are connected by an edge if the films early 1960s by Pieter W. Kasteleyn and, elimination. With these methods, all share an actor. The task is to identify independently, Michael E. Fisher and but a polynomial number of terms in a set of edges linking each vertex to H. N. V. Temperley. It has come to be that giant summation magically cancel exactly one other vertex. The brute- known as the FKT algorithm. out. We never have to compute them, force method of trying all possible The mathematics behind the FKT al- or even look at them. matchings is exponential, but if you gorithm takes some explaining. In out- (The answer sought in the perfect- are given a candidate solution, you can line, the idea is to encode the structure matching problem is actually not the efficiently verify its correctness: of an n-vertex graph in an n-by-n ma- determinant but a related quantity trix; then the number of perfect match- called the Pfaffian. However, the Pfaf- ings is given by an easily computed fian is equal to the square root of the property of the matrix. The illustration determinant, and so the computational on this page shows how the graph is procedure is essentially the same.) represented in matrix form. The existence of a shortcut for evalu- The computation performed on the ating determinants and Pfaffians is like Thus the perfect-matching problem matrix is essentially the evaluation of a a loophole in the tax code—­a windfall lies in NP. determinant. By definition, a determi- for those who can take advantage of it, In the 1960s Jack Edmonds, now of nant is a sum of n! terms, where each but you can only get away with such the University of Waterloo, devised an term is a product of n elements chosen special privileges if you meet very efficient algorithm that finds a perfect from the matrix. The symbol n! denotes stringent conditions. matching if there is one. The Edmonds the factorial of n, or in other words Closely related to the determinant algorithm works in polynomial time, n×(n–1)×...×3×2×1. The trouble is, is another quantity associated with www.americanscientist.org © 2008 Brian Hayes. Reproduction with permission only. 2008 January–February 11 Contact [email protected]. #P-complete. (It was in this work that tions. For certain counting problems, the class #P was first defined.) that’s enough. At a higher level, too, the conspiracy A problem with the cryptic name of circumstances that allows perfect #PL-3-NAE-ICE supplies an example matchings to be counted in polynomial of the holographic process. The prob- time seems rather delicate and sensi- lem concerns a planar graph of maxi- tive to details. The algorithm works mum degree three; that is to say, no only for planar graphs; attempts to ex- vertex has more than three edges. Each tend it to larger families of graphs have edge is to be assigned a direction, sub- failed. Even for planar graphs, it works ject to the constraint that no vertex of only for perfect matchings; counting degree two or three can have all of its the total number of matchings is a #P- edges directed either inward or out- complete task. ward. Graphs of this general kind have been studied as models of the structure Algorithmic Holography of ice; the vertices represent molecules The engine that drives the FKT algo- and the directed edges are chemical rithm is the linear-algebra shortcut for bonds. The decision version of the evaluating determinants (or Pfaffians) problem asks whether the edges can be Counting the configurations of a structure in polynomial time. This prime mover assigned directions so that the not-all- called three-ice is an example of a problem is harnessed to solve a counting prob- equal constraint is obeyed everywhere. solved in polynomial time by a holograph- lem in another area of mathematics, Here we are interested in the counting ic algorithm. Three-ice is a directed graph namely graph theory. Such translations, version, which asks how many ways (each edge has an arrow attached), and no or “reductions,” from one problem to the constraints can be satisfied. more than three edges can meet at any vertex. another are standard fare in complex- The strategy is to build a new pla- Where two or three edges come together, the arrows must not all be either converging or ity theory. Holographic algorithms also nar graph called a matchgrid, which diverging. One valid configuration is shown rely on reductions, and indeed they encodes both the structure of the ice here; the algorithm computes the number of ultimately translate problems into the graph and the not-all-equal constraints ways the arrows can be placed while satisfy- language of determinants. But the na- that have to be satisfied at each vertex. ing the not-all-equal constraint. ture of the reductions is novel. Then we calculate a weighted sum of A typical non-holographic reduc- the perfect matchings in the match- matrices called the permanent. It’s tion is a one-to-one mapping between grid, using the efficient FKT algorithm. another sum of n! products, but even problems in two domains. If you can Although there may be no one-to-one simpler. For the determinant, a compli- reduce problem A to problem B, and mapping between individual match- cated rule assigns positive and nega- then find a solution to an instance of ings in the matchgrid and valid as- tive signs to the various terms of the B, you know that the corresponding signments of bond directions in the ice summation. For the permanent, there’s instance of A also has a solution. De- graph, the weighted sum of the perfect no need to bother keeping track of the vising transformations that set up this matchings is equal to the number of signs; they’re all positive. But the al- one-to-one linkage between problems valid assignments. ternation of signs is necessary for the is a demanding art form. Holographic The matchgrid is constructed from cancellations that allow fast computa- reductions exploit a broader class of components called matchgates, which tion of determinants. As a result, the transformations that don’t necessarily are planar graph fragments that act polynomial loophole doesn’t work for link individual problem instances, but much like the logic gates of Boolean permanents. In 1979 Valiant showed the reductions do preserve the number circuits. To understand how this com- that the calculation of permanents is of solutions or the sum of the solu- puter built of graphs works, it helps to consider first an idealized and simpli- fied version, in which we can manu-    /"&  facture matchgates to meet any speci-  fications we please.    /"&       Given such an unlimited stock of  


   gates, we can assemble a model of PL-  /"&    3-NAE-ICE as follows. A degree-three  vertex in the ice graph is represented   /"&   by a “recognizer” matchgate with three inputs. The recognizer has a “signa- Matchgates are graph-theory devices analogous to the logic gates of digital circuitry. The gate ture” that implements the not-all-equal shown here implements the not-all-equal function for two inputs. It is a chain of five vertices function: the gate is designed so that connected by four edges; the orange exterior vertices accept the inputs. An input pattern is its contribution to the sum of the per- presented to the gate by removing an orange vertex for each 1 in the input, then checking the fect matchings is 0 if the three inputs remaining graph to see if it accommodates a perfect matching. A 00 input removes neither or- ange vertex; the resulting graph cannot have a perfect matching because it has an odd number are 000 or 111, but the contribution is 1 of vertices. The inputs 01 and 10 each remove one vertex and thus do allow a perfect matching; for any of the other six possible input the 11 input again leaves an odd number of vertices. Although the two-input not-all-equal gate patterns (001, 010, 011, 100, 101, 110). is fairly simple, many other functions—including the analogous gate for three inputs—cannot Each edge of the ice graph is repre- be realized in this direct way. sented in the matchgrid by a “genera-

12 American Scientist, Volume 96 © 2008 Brian Hayes. Reproduction with permission only. Contact [email protected]. tor” matchgate that has two outputs, Why are the methods called holo- and can’t be accomplished with polyno- corresponding to the two ends of the graphic algorithms? Valiant explains mial resources. Valiant writes that “any edge. To capture the idea that an edge that their computational power comes proof of P ≠ NP will need to explain, has a direction—­pointing toward one from the mutual cancellation of many and not only to imply, the unsolvability end and away from the other—­the contributions to a sum, as in the optical of our polynomial systems.” generator contributes 1 to the weight- interference pattern that creates a ho- Finally, there’s the challenge of un- ed sum when the outputs are either logram. This sounds vaguely like the derstanding the algorithms themselves 01 or 10, but the contribution is 0 for superposition principle in quantum at a deeper level. To call them “acci- outputs 00 and 11. computing, and that is not entirely a dental”—­or “exotic,” or “freak,” which In this scheme, the concept of per- coincidence. Valiant’s first publication are other terms that turn up in the lit- fect matching becomes a computa- on the topic, in 2002, was titled “Quan- erature—­suggests that they are sports tional mechanism. If a graph fragment, tum circuits that can be simulated clas- of nature, like weird creatures found packaged up as a matchgate, allows sically in polynomial time.” under a rock and put on exhibit. But a perfect matching, then the gate out- one could also argue, on the contrary, puts a logical 1, or true. If no perfect P or NP, That Is the Question that these algorithms are not at all ac- matching is possible, the output is 0, Do holographic algorithms reveal any- cidental; they are highly engineered or false. This is a novel and interesting thing we didn’t already know about constructions. The elaborate systems way to build a computer, but there’s a the P = NP question? Lest there be any of polynomials needed to create sets of catch. Many of the needed matchgates misunderstanding, one point bears matchgates are not something found cannot be implemented in the direct emphasizing: Although some of the in the primordial ooze of mathematics. manner described above. In particu- problems solved by holographic meth- Someone had to invent them. lar, the three-input not-all-equal gate ods were not previously known to be cannot be implemented. The reason in P, none of them were NP-complete Bibliography is easy to demonstrate. Perfect match- or #P-complete. Thus, so far, the bar- Cai, Jin-Yi. Preprint. Holographic algorithms. ing is all about parity; a graph can- rier between P and NP remains intact. http://pages.cs.wisc.edu/~jyc/papers/ not possibly have a perfect matching Suggesting that P might be equal HA-survey.pdf. Cai, Jin-Yi, Vinay Choudhary and Pinyan Lu. unless the number of vertices is even. to NP is deeply unfashionable. A few 2007. On the theory of matchgate com- The three-input not-all-equal gate is years ago William Gasarch of the Uni- putations. In Proceedings of the 22nd IEEE required to respond in the same way versity of Maryland took a poll on the Conference on Computational Complexity, pp. to the inputs 000 and 111. But these question. Of 100 respondents, only nine 305–318. patterns have opposite parity; one is stood on the side of P = NP, and Gas- Cai, Jin-Yi, and Pinyan Lu. 2007. Holographic algorithms: From art to science. In Proceed- even and the other is odd. arch reported that some of them took ings of the 39th ACM Symposium on the Theo- The remedy for this impediment is the position “just to be contrary.” The ry of Computing, STOC ’07, pp. 401–410. yet more linear algebra. Although no idea that all NP problems have easy Cai, Jin-Yi, and Pinyan Lu. 2007. Holographic simple matchgate can directly imple- solutions seems too good to be true, an algorithms: The power of dimensionality ment the three-input not-all-equal exercise in wishful thinking; it would resolved. In Proceedings of the 34th Interna- tional Colloquium on Automata, Languages and function, the desired behavior can be be miraculous if we lived in a universe Programming, ICALP 2007, pp. 631–642. generated as a linear superposition where computing is so effortless. But Cook, Stephen A. 1971. The complexity of of other functions. Finding an appro- the miracle argument cuts both ways: ­theorem-proving procedures. In Proceedings priate set of equations to create such For NP to remain aloof from P, we have of the Third ACM Symposium on the Theory of superpositions is the most essential to believe that not even one out of all Computing, pp. 151–158. and also the most difficult aspect of ap- those thousands of NP-complete prob- Edmonds, Jack. 1965. Paths, trees, and flowers. Canadian Journal of Mathematics 17:449–467. plying the holographic method. It has lems has an efficient solution. Gasarch, William I. 2002. The P=?NP poll. mostly been a hit-or-miss proposition, Valiant suggests a comparison with SIGACT News 33(2):34–47. requiring both inspiration and expert the Goldbach conjecture, which holds Jerrum, Mark. 2003. Counting, Sampling and use of computer-algebra systems. Cai that every even number greater than Integrating: Algorithms and Complexity. Basel, and Pinyan Lu of Tsinghua University 2 is the sum of two primes. Nearly ev- Switzerland: Birkhauser. have recently made progress on sys- eryone believes it to be true, but in the Kasteleyn, P. W. 1961. The statistics of dimers on a lattice. Physica 27:1209–1225. tematizing the search process. absence of a proof, we don’t know why Mertens, Stephan. 2002. Computational com- So far about a dozen counting prob- it should be true. We can’t rule out the plexity for physicists. Computing in Science lems have been solved by the holo- possibility that exceptions exist but are and Engineering 4(3):31–47. graphic method, none of them having so rare we haven’t stumbled on one Valiant, L. G. 1979. The complexity of comput- any immediate practical use or conse- yet. Likewise with the P and NP ques- ing the permanent. Theoretical Computer Sci- quences for the further development tion: A polynomial algorithm for just ence 8:189–201. Valiant, Leslie G. 2002. Quantum circuits that of complexity theory. Indeed, they are one NP-complete problem would for- can be simulated classically in polynomial an odd lot—­apart from the ice prob- ever alter the landscape. time. SIAM Journal on Computing 31:1229– lem they are mostly specialized and The work on holographic algorithms 1254. restricted versions of satisfiability and doesn’t have to be seen as some sort Valiant, Leslie G. 2005. Holographic algorithms. matching. One particularly curious re- of wildcat drilling expedition, hoping Electronic Colloquium on Computational Com- sult gives a fast algorithm for counting to strike a P = NP gusher. It would be plexity, Report No. 99.l Valiant, Leslie G. 2006. Accidental algorithms. a certain set of solutions modulo 7, but worthwhile just to have a finer survey In Proceedings of the 47th IEEE Symposium on counting the same set modulo 2 is at of the boundaries between complexity Foundations of Computer Science, FOCS ’06, least as hard as NP-complete. classes, showing more clearly what can pp. 509–517. www.americanscientist.org © 2008 Brian Hayes. Reproduction with permission only. 2008 January–February 13 Contact [email protected].