Logic Synthesis Meets Machine Learning: Trading Exactness for Generalization Shubham Raif,6,y, Walter Lau Neton,10,y, Yukio Miyasakao,1, Xinpei Zhanga,1, Mingfei Yua,1, Qingyang Yia,1, Masahiro Fujitaa,1, Guilherme B. Manskeb,2, Matheus F. Pontesb,2, Leomar S. da Rosa Juniorb,2, Marilton S. de Aguiarb,2, Paulo F. Butzene,2, Po-Chun Chienc,3, Yu-Shan Huangc,3, Hoa-Ren Wangc,3, Jie-Hong R. Jiangc,3, Jiaqi Gud,4, Zheng Zhaod,4, Zixuan Jiangd,4, David Z. Pand,4, Brunno A. de Abreue,5,9, Isac de Souza Camposm,5,9, Augusto Berndtm,5,9, Cristina Meinhardtm,5,9, Jonata T. Carvalhom,5,9, Mateus Grellertm,5,9, Sergio Bampie,5, Aditya Lohanaf,6, Akash Kumarf,6, Wei Zengj,7, Azadeh Davoodij,7, Rasit O. Topalogluk,7, Yuan Zhoul,8, Jordan Dotzell,8, Yichi Zhangl,8, Hanyu Wangl,8, Zhiru Zhangl,8, Valerio Tenacen,10, Pierre-Emmanuel Gaillardonn,10, Alan Mishchenkoo,y, and Satrajit Chatterjeep,y aUniversity of Tokyo, Japan, bUniversidade Federal de Pelotas, Brazil, cNational Taiwan University, Taiwan, dUniversity of Texas at Austin, USA, eUniversidade Federal do Rio Grande do Sul, Brazil, fTechnische Universitaet Dresden, Germany, jUniversity of Wisconsin–Madison, USA, kIBM, USA, lCornell University, USA, mUniversidade Federal de Santa Catarina, Brazil, nUniversity of Utah, USA, oUC Berkeley, USA, pGoogle AI, USA The alphabetic characters in the superscript represent the affiliations while the digits represent the team numbers yEqual contribution. Email: [email protected], [email protected], [email protected], [email protected] Abstract—Logic synthesis is a fundamental step in hard- artificial intelligence. A central problem in machine ware design whose goal is to find structural representations learning is that of supervised learning: Given a class of Boolean functions while minimizing delay and area. H of functions from a domain X to a co-domain Y , find If the function is completely-specified, the implementa- tion accurately represents the function. If the function is a member h 2 H that best fits a given set of training incompletely-specified, the implementation has to be true examples of the form (x; y) 2 X × Y . The quality of the only on the care set. While most of the algorithms in logic fit is judged by how well h generalizes, i.e., how well h synthesis rely on SAT and Boolean methods to exactly fits examples that were not seen during training. implement the care set, we investigate learning in logic synthesis, attempting to trade exactness for generalization. Thus, logic synthesis and machine learning are closely This work is directly related to machine learning where related. Supervised machine learning can be seen as logic the care set is the training set and the implementation synthesis of an incompletely specified function with a is expected to generalize on a validation set. We present different constraint (or objective): the circuit must also learning incompletely-specified functions based on the re- sults of a competition conducted at IWLS 2020. The goal generalize well outside the careset (i.e., to the test set) of the competition was to implement 100 functions given possibly at the expense of reduced accuracy on the careset by a set of care minterms for training, while testing the (i.e., on the training set). Conversely, logic synthesis may implementation using a set of validation minterms sampled be seen as a machine learning problem where in addition from the same function. We make this benchmark suite to generalization, we care about finding an element of H available and offer a detailed comparative analysis of the different approaches to learning. that has small size, and the sets X and Y are not smooth but discrete. I. INTRODUCTION To explore this connection between the two fields, the arXiv:2012.02530v2 [cs.LG] 15 Dec 2020 two last authors of this paper organized a programming Logic synthesis is a key ingredient in modern electronic contest at the 2020 International Workshop in Logic design automation flows. A central problem in logic Synthesis. The goal of this contest was to come up synthesis is the following: Given a Boolean function with an algorithm to synthesize a small circuit for a f : n ! (where denotes the set f0; 1g), construct B B B Boolean function f : Bn ! B learnt from a training set a logic circuit that implements f with the minimum of examples. Each example (x; y) in the training set is an number of logic gates. The function f may be completely n n input-output pair, i.e., x 2 B and y 2 B. The training set specified, i.e., we are given f(x) for all x 2 B , or it may was chosen at random from the 2n possible inputs of the be incompletely specified, i.e., we are only given f(x) n n function (and in most cases was much smaller than 2 ). for a subset of B called the careset. An incompletely The quality of the solution was evaluated by measuring specified function provides more flexibility for optimizing accuracy on a test set not provided to the participants. the circuit since the values produced by the circuit outside the careset are not of interest. The synthesized circuit for f had to be in the form Recently, machine learning has emerged as a key of an And-Inverter Graph (AIG) [1, 2] with no more enabling technology for a variety of breakthroughs in than 5000 nodes. An AIG is a standard data structure used in logic synthesis to represent Boolean functions This work was supported in part by the Semiconductor Research where a node corresponds to a 2-input And gate and Corporation under Contract 2867.001. edges represent direct or inverted connections. Since an Table I: An overview of different types of functions in AIG can represent any Boolean function, in this problem the benchmark set. They are selected from three domains: H is the full set of Boolean functions on n variables. Arithmetic, Random Logic, and Machine Learning. To evaluate the algorithms proposed by the participants, 00-09 2 MSBs of k-bit adders for k 2 f16; 32; 64; 128; 256g we created a set of 100 benchmarks drawn from a 10-19 MSB of k-bit dividers and remainder circuits for k 2 f16; 32; 64; 128; 256g mix of standard problems in logic synthesis such as 20-29 MSB and middle bit of k-bit multipliers for k 2 f8; 16; 32; 64; 128g synthesis of arithmetic circuits and random logic from 30-39 k-bit comparators for k 2 f10; 20;:::; 100g standard logic synthesis benchmarks. We also included 40-49 LSB and middle bit of k-bit square-rooters with k 2 f16; 32; 64; 128; 256g 50-59 10 outputs of PicoJava design with 16-200 inputs and roughly balanced onset & offset some tasks from standard machine learning benchmarks. 60-69 10 outputs of MCNC i10 design with 16-200 inputs and roughly balanced onset & offset For each benchmark the participants were provided with 70-79 5 other outputs from MCNC benchmarks + 5 symmetric functions of 16 inputs the training set (which was sub-divided into a training set 80-89 10 binary classification problems from MNIST group comparisons proper of 6400 examples and a validation set of another 90-99 10 binary classification problems from CIFAR-10 group comparisons 6400 examples though the participants were free to use these subsets as they saw fit), and the circuits returned by their algorithms were evaluated on the corresponding false value otherwise. The threshold value is defined test set (again with 6400 examples) that was kept private during training. Hence, each internal node can be seen until the competition was over. The training, validation as a multiplexer, with the selector given by the threshold and test sets were created in the PLA format [3]. The value. Random forests are composed by multiple decision score assigned to each participant was the average test trees, where each tree is trained over a distinct feature, accuracy over all the benchmarks with possible ties being so that trees are not very similar. The output is given by broken by the circuit size. the combination of individual predictions. Ten teams spanning 6 countries took part in the contest. Look-up Table (LUT) Network is a network of They explored many different techniques to solve this randomly connected k-input LUTs, where each k-input problem. In this paper we present short overviews of the LUT can implement any function with up to k variables. techniques used by the different teams (the superscript LUT networks were first employed in a theoretical for an author indicates their team number), as well a study to understand if pure memorization (i.e., fitting comparative analysis of these techniques. The following without any explicit search or optimization) could lead are our main findings from the analysis: to generalization [11]. • No one technique dominated across all the bench- III. BENCHMARKS marks, and most teams including the winning team used an ensemble of techniques. The set of 100 benchmarks used in the contest can • Random forests (and decision trees) were very be broadly divided into 10 categories, each with 10 test- popular and form a strong baseline, and may be cases. The summary of categories is shown in Table I. For a useful technique for approximate logic synthesis. example, the first 10 test-cases are created by considering k • Sacrificing a little accuracy allows for a significant the two most-significant bits (MSBs) of -input adders reduction in the size of the circuit. for k 2 f16; 32; 64; 128; 256g.