Harnessing Simulation Acceleration to Solve the Digital Design Verification Challenge by Debapriya Chatterjee A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science and Engineering) in The University of Michigan 2013 Doctoral Committee: Associate Professor Valeria M. Bertacco, Chair Professor Todd M. Austin Professor Igor L. Markov Assistant Professor Zhengya Zhang c Debapriya Chatterjee All Rights Reserved 2013 To my parents ii Acknowledgments I would like to thank my advisor Professor Valeria Bertacco, who introduced me to re- search. Her willingness to let me explore new ideas has been a cherished freedom, while our discussions on research directions have enabled me to pursue a fixed direction in an interesting and uncharted research territory. Moreover, my writing and presentation skills have grown significantly as a result of her mentoring and her careful attention. I am also grateful for my committee members. I am appreciative of Professor Igor Markov’s contributions: often pointing me to relevant research articles as well as our friendly interactions in the department and various conferences. Professor Todd Austin has been a consistent source of extremely valuable feedback on various matters throughout my studies. I am grateful to Professor Zhengya Zhang for his kind feedback. Early in my graduate school career, I was very fortunate to work with Ilya Wagner and Andrew DeOrio – they instilled the fundamentals of the role of a graduate student into me. I am especially grateful to Joseph Greathouse, Andrea Pellegrini and Biruk Mammo for the innumerable illuminating discussions we have had in the office, often extending into late hours. I am indebted to Ronny Morad, Amir Nahir, Avi Ziv and Anatoly Koyfman from IBM Research, Haifa for allowing me to engage in a very close collaboration with the in- dustry. I am really appreciative of the people I have collaborated with on many projects over the years: Andrew DeOrio, Calvin McCarter, Biruk Mammo, Chang-Hong Hsu, Doowon Lee, Sara Vinco, Nicola Bombieri, Ronny Morad, Amir Nahir, Avi Ziv, Anatoly Koyfman, Raviv Gal, Dmitry Pidan and Professor Franco Fummi. I am also appreciative of the in- teresting and insightful discussions with my colleagues Rawan Abdel-Khalek and Ritesh Parikh. I would like to thank Dhanajay Anand, my friend and apartment mate for the entire duration of my stay in Ann Arbor, who supported me through thick and thin. I would also like to thank my friend from college days, Shaunak Chatterjee, for motivating me on many occasions. Finally, I would like to thank my parents: for providing me emotional support over half a decade halfway across the globe. iii Preface Today, design verification is by far the most resource and time-consuming activity in the hardware development process for complex commercial integrated circuit designs such as microprocessor and system-on-chip (SoC) designs. Within this area, the vast majority of the verification effort in industry relies on simulation platforms, which can be implemented either in hardware or software. A “simulator” includes a model of each component of a design and has the capability of simulating its behavior under any input scenario provided by an engineer. Thus, simulators are deployed to evaluate the behavior of a design under as many input scenarios as possible and to identify and debug all incorrect functionality. Two features are critical in simulators for the validation effort to be effective: performance and checking/debugging capabilities. A wide range of simulator platforms are available to- day: on one end of the spectrum there are software-based simulators, providing a very rich software infrastructure for checking and debugging the design’s functionality, but execut- ing only at 1-10 simulation cycles per second (compare this to actual silicon chips, which execute at GHz speeds). At the other end of the spectrum, there are hardware-based plat- forms, such as accelerators, emulators and even prototype silicon chips, providing higher performances by 4 to 9 orders of magnitude, at the cost of very limited or non-existent checking/debugging capabilities. As a result, today, simulation-based validation is crip- pled: one can either have satisfactory performance on hardware-accelerated platforms or critical infrastructures for checking/debugging on software simulators, but not both. This dissertation brings together the two ends of this spectrum by providing high perfor- mance solutions for software-based platforms and quality checking/debugging capabilities for hardware-based verification systems. To this end, the dissertation uses a two-pronged approach: it infuses performance into software simulators, and it brings in checking and debugging capabilities into hardware-based platforms. Specifically, it addresses the perfor- mance challenge of software simulators by leveraging inexpensive off-the-shelf graphics processors as a massively parallel execution substrate, and then exposing to it the paral- lelism inherent in the design model. The outcome is a simulation solution that achieves an order of magnitude speedup over traditional software-based simulation. For hardware- iv based platforms, the dissertation provides solutions that offer enhanced checking and debugging capabilities by abstracting the relevant data to be logged during simulation so to minimize the cost of collection, transfer and processing. Observability for check- ing/debugging on these platforms is improved with a state restoration solution that is capable of reconstructing a number of signals while observing only a small fraction of them. Along with improved observability, the dissertation also brings in a number of high quality checking capabilities, comparable to those of their software-based simulation coun- terparts, while only imposing minimal logic or performance overhead (∼20%). Altogether, the contributions of this dissertation enable effective high-performance simulation-based validation. v Table of Contents Dedication ....................................... ii Acknowledgments ................................... iii Preface ......................................... iv List of Figures ..................................... x List of Tables ...................................... xii Chapter 1 Introduction ............................... 1 1.1 Importanceofverificationinthedesignflow . ...... 2 1.2 Phasesoffunctionalverification . .... 6 1.2.1 Pre-siliconverification . 7 1.2.2 Post-siliconvalidation . 8 1.3 Factors cripplingsimulation-basedvalidation . .......... 9 1.3.1 Limited performance of software-based simulation . ....... 10 1.3.2 Limited validation capability for hardware-accelerated simulation . 10 1.3.3 Thesimulation-basedvalidationgap . ... 11 1.4 Overviewofmydissertation . 12 1.4.1 Bridgingthesimulation-basedvalidationgap . ...... 12 1.4.2 Improving performance of software-based simulation ........ 13 1.4.3 Bringing in validation capability to hardware-accelerated platforms 14 1.5 Organizationofthedissertation. ..... 16 Chapter 2 The Simulation Spectrum ........................ 18 2.1 Spectrumofvalidationplatforms . .... 19 2.1.1 Software-basedsimulation . 20 2.1.2 Accelerationplatform . 21 2.1.3 Emulationplatform. 22 2.1.4 Siliconprototype . .. .. .. .. .. .. .. 23 2.2 State-of-the-art in high-performance simulation-basedvalidation . 24 2.2.1 Synthesizingcheckingconstructs . .. 25 2.2.2 Tracingsignalsforoff-linechecking . .... 25 vi 2.2.3 Observabilityviareconstruction . ... 26 2.2.4 Replayfromstatesnapshot. 27 2.3 Keychallenges ................................ 27 2.4 Contributions ................................. 29 2.4.1 Infusing performance into software-based simulation........ 29 2.4.2 Providingobservabilitythroughrestoration . ....... 30 2.4.3 Enabling checking capability in hardware-acceleratedplatforms . 30 Chapter 3 The Quest for Simulation Speed .................... 33 3.1 High-performance simulation through massive parallel processing . 33 3.1.1 Overviewofthischapter . 35 3.2 Introduction to GP-GPU architecture and programming model ....... 36 3.3 Towardshigh-performancelogicsimulation . ....... 38 3.4 Oblivioussimulatoroverview. ... 40 3.4.1 Synthesis and combinational netlist extraction . ........ 41 3.4.2 Clustering............................... 41 3.4.3 Clusterbalancing. .. .. .. .. .. .. .. 43 3.4.4 Simulation .............................. 43 3.5 Event-drivensimulatoroverview . .... 44 3.5.1 Segmentationintomacro-gates. 45 3.5.2 Macro-gatebalancing. 46 3.5.3 Simulationphase ........................... 47 3.6 GCSexperimentalresults. 49 3.6.1 Performanceoftheoblivioussimulator . ... 50 3.6.2 Performanceoftheevent-drivensimulator . ..... 50 3.7 Towards high-performance behavioral simulation . ......... 51 3.8 MappingSystemCtoGP-GPU . 53 3.8.1 Constructionofprocessdependencygraph . ... 53 3.8.2 Partitioningintoconcurrentdataflows . .... 55 3.8.3 ParallelexecutioninCUDA . 56 3.9 SAGAexperimentalevaluation . .. 58 3.9.1 Experimentalsetup . 58 3.9.2 Performance ............................. 60 3.9.3 Architecturecomparison . 61 3.10Relatedwork ................................. 61 3.11Summary ................................... 63 Chapter 4 Providing Observability for Hardware-accelerated Simulation ... 65 4.1 Towards obtaining observability beyond software-basedsimulation . 65 4.1.1 Overviewofthischapter . 67 4.2 Backgroundofstaterestoration. .... 67 4.3 Structure of existing signal selection algorithms . ........... 69 4.3.1 The problem of diminishing