Algorithms for Circuits and Circuits for Algorithms (Invited Paper) Ryan Williams Computer Science Department Stanford University Stanford, CA, USA Email: [email protected] Abstract—The title of this paper is meant to highlight an of consequences of such impossibility results, as well as emerging duality between two fundamental topics in algorithms consequences of possessing such interesting methods (in the and complexity theory. hopes of an eventual proof by contradiction). Algorithms for circuits refers to the design of interesting algorithms which can perform non-trivial circuit analysis of Another dictum is that algorithm design and analysis is, some kind, on either a circuit or a Boolean function given as a on the whole, an easier venture than proving lower bounds. truth table. For instance, an algorithm determining whether a Conventional wisdom states that in algorithm design, one given circuit has an input that forces a true output would solve only has to find a single efficient algorithm that will solve the NP-complete Circuit-SAT problem. Such an algorithm is of the problem at hand, but a lower bound must reason about all course unlikely to run in polynomial time, but could possibly be more efficient than exhaustively trying all possible inputs possible efficient algorithms, including bizarrely behaving to the circuit. ones, and argue that none solve the problem at hand. This Circuits for algorithms refers to the modeling of uniform dictum is also reflected in the literature: every year, many in- algorithms with non-uniform circuit families (or proving such teresting algorithms are discovered, analyzed, and published, modeling is impossible). For instance, the NEXP versus P=poly compared to the tiny number of lower bounds proved.1 question asks whether nondeterministic exponential-time algo- Furthermore, there are theoretical reasons for believing that rithms can be simulated using non-uniform circuit families of polynomial size. It is widely believed that the answer is no, lower bounds are hard to prove. The most compelling of however the present mathematical tools available are still too these are the three “barriers” of Relativization [BGS75], crude to prove this kind of separation. Natural Proofs [RR97], and Algebrization [AW09]. These This paper surveys these two generic subjects, the ways in “no-go” theorems demonstrate that the known lower bound which they arise, and connections that have been developed proof methods are simply too coarse to prove even weak between them, focusing on the connections between non- lower bounds, much weaker than P 6= NP. Subsequently, trivial circuit-analysis algorithms and proofs of circuit size lower bounds. To give one example, if there is a nontrivial complexity theory has been clouded with great pessimism algorithm (running slightly faster than exhaustive search) that about resolving some of its central open problems. can determine if a given circuit computes a constant function, While the problems of algorithm design and proving then it can be concluded that NEXP is not contained in P=poly. lower bounds may arise from looking at opposing tasks, Informally, this connection can be interpreted as saying “some good algorithms for circuits imply there are no good circuits the two tasks have deep similarities when viewed in the for some algorithms.” appropriate way.2 This survey will concentrate on some of the most counterintuitive similarities: from the design of Keywords-satisfiability; derandomization; exact algorithms; learning; circuit complexity; parameterized algorithms certain algorithms (the supposedly “easier” task), one can derive new lower bounds (the supposedly “harder” task). I. INTRODUCTION That is, there are senses in which algorithm design is at least as hard as proving lower bounds. Such implications Budding theoretical computer scientists are generally present an excellent mathematical “arbitrage” opportunity taught several dictums at an early age. One such dictum for complexity theorists, to potentially prove hard lower is that the algorithm designers and the complexity theorists bounds via supposedly easier algorithm design. (Moreover, (whoever they may be) are charged with opposing tasks. The this approach has recently led to new lower bounds.) algorithm designer discovers interesting methods for solving Some connections take the following form. Suppose there certain problems; along the way, she may also propose new notions of what is interesting, to better understand the 1Of course, there can be other reasons for this disparity, such as funding. scope and power of algorithms. The complexity theorist is 2Similarities can already be found in the proof(s) that the Halting Prob- supposed to prove lower bounds, showing that sufficiently lem is undecidable: the workhorse behind such results is the construction of a universal Turing machine that can run arbitrary Turing machine code interesting methods for solving certain problems do not given as input. This is a canonical example of a positive algorithmic result exist. Barring that, he develops a structural framework applied to prove an impossibility result. is a Turing machine T which receives on its input tape a circuit families. Section IV surveys existing knowledge of description of a finite logical circuit C, and on all “struc- circuit-analysis algorithms, which we call algorithms for tured” circuits C, T is guaranteed to perform some nontrivial circuits. Section V discusses known connections between analysis of the function computed by C. (For example, T the two, and prospects for future progress. Section VI briefly could determine whether C outputs the same value on all concludes. possible inputs to C, provided C is a “shallow” circuit with a T few layers of gates.) Such a can then be used to construct II. PRELIMINARIES a function f that is computable “somewhat efficiently” by a Turing machine but is not computable efficiently by non- We assume familiarity with machine-based complexity uniform circuit families possessing that structure. That is, an theory [AB09] but not necessarily circuit complexity. interesting circuit-analysis algorithm can be applied to prove Circuit complexity is concerned with how to construct an interesting circuit complexity lower bound. Boolean functions out of “simpler” functions, such as those It is worth emphasizing the quantifiers in the above of the form g : f0; 1g2 ! f0; 1g. Examples of Boolean implication schema: functions include: The existence of an algorithm T that can analyze all structured circuits C, implies the existence of a function f • ORk(x1; : : : ; xk), ANDk(x1; : : : ; xk), with their usual that is not computable by all structured circuit families. logical meanings, • MODm (x ; : : : ; x ) for a fixed integer m > 1, which That is, there are situations in which designing algorithms k 1 k outputs 1 if and only if P x is divisible by m. for some problem X can be translated into “lower bound i i • MAJ (x ; : : : ; x ) = 1 if and only if P x ≥ dk=2e. design” for another problem Y . The key is that there k 1 k i i are two computational models under consideration here: Circuit complexity: A basis set B is a set of Boolean the algorithm model or the usual “Turing” style model of functions. Two popular choices for B are B2, the set of all 2 algorithms, and the circuit model or the non-uniform circuit functions g : f0; 1g ! f0; 1g, and U2, the set B2 without family model. Careful design of algorithms for analyzing MOD2 and the negation of MOD2. A Boolean circuit of size given instances of the circuit model are used to construct s with n inputs x1; : : : ; xn over basis B is a sequence of n+s n functions computable (in one sense) in the algorithm model functions C = (f1; : : : ; fn+s), with fi : f0; 1g ! f0; 1g that are uncomputable (in another sense) in the circuit model. for all i, such that: There is a kind of duality lurking beneath which is not well- • i = 1; : : : ; n f (x ; : : : ; x ) = x understood. for all , i 1 n i, • for all j = n + 1; : : : ; n + s, there is a func- This article will survey two generic topics in algorithms tion g : f0; 1gk ! f0; 1g from B and in- and complexity, and connections between them: dices i1; : : : ; ik < j such that fj(x1; : : : ; xn) = • Circuits for algorithms refers to the modeling of pow- g(fi1 (x1; : : : ; xn); : : : ; fik (x1; : : : ; xn)). erful uniform algorithms with non-uniform circuit fam- ilies (or proving that such modeling is impossible). The fi are called the gates of the circuit; f1; : : : ; fn are For instance, the EXP versus P=poly question asks the input gates, fn+1; : : : ; fn+s−1 are the internal gates, whether exponential-time algorithms can be simulated and fn+s is the output gate. The circuit C can naturally be using non-uniform circuit families of polynomial size. thought of as a function as well: on an input string x = n • Algorithms for circuits refers to designing interesting (x1; : : : ; xn) 2 f0; 1g , C(x) denotes fn+s(x). algorithms which can perform some interesting circuit Thinking of the connections between the gates as a analysis. The input may be a circuit or it may be directed acyclic graph in the natural way, with the input gates a Boolean function (given as a truth table), and the as n source nodes 1; : : : ; n, and the jth gate with indices algorithm checks whether the circuit (or truth table) sat- i1; : : : ; ik < j as a node j with incoming arcs from nodes isfies a simple property related to the circuit complexity i1; : : : ; ik, the depth of C is the longest path from an input of the underlying function. To illustrate, an algorithm gate to the output gate. As a convention, we will not count determining if a given circuit has an input that forces gates with fan-in 1 in the depth measure. That is, gates of a true output solves the NP-complete Circuit-SAT the form g(x) = x or g(x) = :x are not counted towards problem.