The Work of Leslie Valiant

Avi Wigderson School of Mathematics Institute for Advanced Study Princeton, New Jersey [email protected]

ABSTRACT Model and its variants. This led to well-defined research On Saturday, May 30, one day before the start of the regu- directions and created a language that enabled this study lar STOC 2009 program, a workshop was held in celebra- to mature into a real scientific theory, with techniques, re- tion of Leslie Valiant’s 60th birthday. Talks were given sults and connections to the rest of . The by Jin-Yi Cai, Stephen Cook, Vitaly Feldman, Mark Jer- resulting theory turned out to have numerous applications rum, Michael Kearns, Mike Paterson, Michael Rabin, Rocco in a very wide area of computer practice (e.g., in Computer Servedio, Paul Valiant, Vijay Vazirani, and Avi Wigder- Vision and Natural Language Processing) as well as inform son. The workshop was organized by Michael Kearns, Rocco other scientific disciplines (e.g., the study of proteins and Servedio, and Salil Vadhan, with support from the STOC lo- the cognitive functions of the brain). It is impossible to give cal arrangements team and program committee. details here of Valiant’s numerous concrete contributions to To accompany the workshop, here we briefly survey Valiant’s computational learning theory. We only mention a recent many fundamental contributions to the theory of comput- work of his that attempts to present a computational theory ing. of evolution, which builds on his learning framework [20].

Categories and Subject Descriptors 3. COMPUTATIONAL COMPLEXITY A.0 [General Literature]: Biographies/autobiographies; Valiant’s seminal contributions to this field are numerous, F.0 []: General and we will touch on only a few of them. We start with his definition of the class #P of counting problems [8], and his proof that the Permanent is complete for this class [7]. Be- General Terms yond the direct interest in this class, these innovations have Theory served computational complexity in unforeseen ways. For example, the class #P was shown to contain the polynomial 1. INTRODUCTION time hierarchy [3]. Moreover, the special properties of Per- manent, such as random-self-reducibility, played a key role Les Valiant singlehandedly created, or completely trans- in unveiling the power of interactive proofs, PCPs, program formed, several fundamental research areas of computer sci- checking and more. Next, Valiant’s algebraic analogue of ence. Here we list some of those areas, and some of Valiant’s Boolean complexity (for the computation of polynomials) [6] seminal contributions to each. served as the pivot for much subsequent research. Valiant’s own seminal contributions to this research include identi- 2. COMPUTATIONAL LEARNING THEORY fying the Permanent and Determinant as complete prob- Valiant’s “Theory of the Learnable” [13] created the area lems for two natural classes of algebraic computations [6], of Computational Learning Theory. Mainstream research proving that (in this model) negation can be exponentially in today has embraced Valiant’s view- more powerful than monotone computation [9], and last but point as a path to understand and design intelligent sys- not least, surprisingly proving that sequential computation tems. Philosophically, Valiant’s work presents a computa- can always be parallelized (in contrast to what we believe tional theory that mirrors or at least serves as a metaphor for to be true in Boolean complexity) [21]. Valiant has also one of the most basic human activities: learning. Pragmat- made numerous contributions to the study of Boolean cir- ically, this work suggests a framework for the development cuit complexity. In particular, the fundamental concepts of of that adapt their behavior in response to feed- superconcentrators (see below) and rigidity have originated back from the environment. Needless to say, attempts to from his works [4, 5], and still serve as some of the very model the learning process were made before Valiant. His few tools for proving (weak, so far) time-space trade-offs contribution was in providing a general framework, as well as and circuit lower bounds. We mention that superconcentra- very concrete computational models for studying questions tors were also the inspiration (in a precise technical sense) relating to the learning process, namely the PAC Learning for the first construction of error-correcting codes that are optimal to within constant factors in all aspects: linear dis- tance, constant rate, and linear-time encoding and decod- Copyright is held by the author/owner(s). STOC’09, May 31–June 2, 2009, Bethesda, Maryland, USA. ing [2]. Valiant’s contributions have also had a significant ACM 978-1-60558-506-2/09/05. impact on other areas of complexity theory. The list includes

1 his polynomial-size monotone formula for majority [12], the a submodel of the standard quantum Turing machine that study of the complexity of finding unique solutions [22], and has a classical simulation in polynomial time [16]. The ram- the relation between approximate counting and uniform gen- ifications of these ideas, both as a new classical algorithmic eration [1]. technique as well as a quantum computational model are currently being pursued by a number of researchers (as was 4. PARALLEL AND DISTRIBUTED COM- the “fate” of many of Valiant’s earlier works). PUTATION 7. SUMMARY Perhaps the most striking single contribution to this area Valiant has made enormous contributions to computer sci- is Valiant’s randomized distributed routing (and ence; his work has had a huge impact on both its theory and its beautiful analysis) [11]. Randomized routing provides practice, as well as the embodiment of computational princi- a way of avoiding congestion in sparse networks, something ples in other scientific disciplines. We all became addicted unavoidable for any deterministic algorithm. This algorithm to this remarkable throughput, and expect more. Happy and its follow-ups have become basic tools in parallel and birthday Les! distributed architectures, memory management and load bal- ancing. In a survey on parallel computing in the early 1980s [10], Valiant posed several challenges that shaped the 8. REFERENCES direction of this field. These challenges called for at- [1] M. R. Jerrum, L. G. Valiant, and V. V. Vazirani. Random tempting to parallelize some seemingly “inherently sequen- generation of combinatorial structures from a uniform tial” problems like “maximal independent set” and “maxi- distribution. Theoret. Comput. Sci., 43(2-3):169–188, 1986. mum matching”, and addressing them gave rise to the devel- [2] D. A. Spielman. Linear-time encodable and decodable error-correcting codes. IEEE Trans. Inform. Theory, 42(6, opment of some of the most basic algorithmic techniques we part 1):1723–1731, 1996. have in this field. Another such basic technique is presented [3] S. Toda. PP is as hard as the polynomial-time hierarchy. in Valiant’s parallelization of polynomial computation (men- SIAM J. Comput., 20(5):865–877, 1991. tioned above) [21]. In addition, the aforementioned “super- [4] L. G. Valiant. Graph-theoretic properties in computational concentrators”, which Valiant explicitly and optimally con- complexity. J. Comput. System Sci., 13(3):278–285, 1976. structed in his work on circuit complexity [4], serve as funda- [5] L. G. Valiant. Graph-theoretic arguments in low-level mental networks, which in turn underlie distributed sorting, complexity. In Proc. 6th MFCS, pp. 162–176. Springer searching and routing algorithms. Finally, Valiant’s Bulk- LNCS 53, Berlin, 1977. [6] L. G. Valiant. Completeness classes in algebra. In Proc. Synchronous Parallel (BSP) Model bridges the gap between 11th STOC, pp. 249–261, 1979. software and hardware for parallel computation, enabling [7] L. G. Valiant. The complexity of computing the arbitrary parallel algorithms to be efficiently compiled onto permanent. Theoret. Comput. Sci., 8(2):189–201, 1979. a wide variety of parallel architectures [14]. [8] L. G. Valiant. The complexity of enumeration and reliability problems. SIAM J. Comput., 8(3):410–421, 1979. [9] L. G. Valiant. Negation can be exponentially powerful. 5. COGNITIVE THEORIES Theoret. Comput. Sci., 12(3):303–314, 1980. Valiant has been fascinated with the workings of the brain. [10] L. G. Valiant. Parallel computation. In Proceedings of the Following his work on computational learning theory, he 7th IBM Symposium on Mathematical Foundations of tried to explain how the (assumed) architecture of the brain Computer Science, 1982. can support complex cognitive functions that humans seem [11] L. G. Valiant. A scheme for fast parallel communication. to perform trivially, like memory, complex pattern recogni- SIAM J. Comput., 11(2):350–361, 1982. [12] L. G. Valiant. Short monotone formulae for the majority tion, natural language recognition, rule learning and more. function. J. Algorithms, 5(3):363–366, 1984. His book “Circuits of the Mind” [15] offers a concrete math- [13] L. G. Valiant. A theory of the learnable. Commun. ACM, ematical model of the brain, together with programs that 27(11):1134–1142, 1984. (under this architecture) actually perform such cognitive [14] L. G. Valiant. A bridging model for parallel computation. functions. Valiant has continued this research further in Commun. ACM, 33(8):103–111, 1990. a body of subsequent work that elaborates his theory, re- [15] L. G. Valiant. Circuits of the Mind. Oxford University lates it to ongoing discoveries in neuroscience, and provides Press, 1994. experimental support for the theory (e.g. [17, 18]). [16] L. G. Valiant. Quantum circuits that can be simulated classically in polynomial time. SIAM J. Comput., 31(4):1229–1254, 2002. 6. HOLOGRAPHIC ALGORITHMS [17] L. G. Valiant. Memorization and association on a realistic In a recent set of papers (e.g. [19]), Valiant suggests a neural model. Neural Comput., 17(3):527–555, 2005. [18] L. G. Valiant. A quantitative theory of neural novel algorithmic technique, which is able to solve counting computation. Biological Cybernetics, 95(3):205–211, 2006. problems and constraint satisfaction problems that appear [19] L. G. Valiant. Holographic algorithms. SIAM J. Comput., to be inaccessible by any previous method. Abstractly, these 37(5):1565–1594, 2008. algorithms provide reductions to a problem in P, specifically [20] L. G. Valiant. Evolvability. J. ACM, 56(1), 2009. the problem of counting the number of perfect matchings in [21] L. G. Valiant, S. Skyum, S. Berkowitz, and C. Rackoff. planar graphs. However, the reductions are “holographic”, Fast parallel computation of polynomials using few namely they are many-to-many (as opposed to the stan- processors. SIAM J. Comput., 12(4):641–644, 1983. dard many-to-one reductions which are commonly used), [22] L. G. Valiant and V. V. Vazirani. NP is as easy as detecting and seem to use cancellation (i.e., negation) in a mysteri- unique solutions. Theoret. Comput. Sci., 47(1):85–93, 1986. ous way. If this reminds you of quantum computation, it should! Valiant showed that his model may be viewed as

2