MERRIMAC – HIGH-PERFORMANCE AND HIGHLY-EFFICIENT SCIENTIFIC COMPUTING WITH STREAMS A DISSERTATION SUBMITTED TO THE DEPARTMENT OF ELECTRICAL ENGINEERING AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Mattan Erez May 2007 c Copyright by Mattan Erez 2007 All Rights Reserved ii I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. (William J. Dally) Principal Adviser I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. (Patrick M. Hanrahan) I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. (Mendel Rosenblum) Approved for the University Committee on Graduate Studies. iii iv Abstract Advances in VLSI technology have made the raw ingredients for computation plentiful. Large numbers of fast functional units and large amounts of memory and bandwidth can be made efficient in terms of chip area, cost, and energy, however, high-performance com- puters realize only a small fraction of VLSI’s potential. This dissertation describes the Merrimac streaming supercomputer architecture and system. Merrimac has an integrated view of the applications, software system, compiler, and architecture. We will show how this approach leads to over an order of magnitude gains in performance per unit cost, unit power, and unit floor-space for scientific applications when compared to common scien- tific computers designed around clusters of commodity general-purpose processors. The dissertation discusses Merrimac’s stream architecture, the mapping of scientific codes to effectively run on the stream architecture, and system issues in the Merrimac supercom- puter. The stream architecture is designed to take advantage of the properties of modern semiconductor technology — very high bandwidth over short distances and very high transistor counts, but limited global on-chip and off-chip bandwidths — and match them with the characteristics of scientific codes — large amounts of parallelism and access locality. Organizing the computation into streams and exploiting the resulting locality using a register hierarchy enables a stream architecture to reduce the memory bandwidth required by representative computations by an order of magnitude or more. Hence a processing node with a fixed memory bandwidth (which is expensive) can support an order of magnitude more arithmetic units (which are inexpensive). Because each node has much greater performance (128 double-precision GFLOP/s) than a conventional microprocessor, a streaming supercomputer can achieve a given level of performance with fewer nodes, reducing costs, simplifying system management, and increasing reliability. v Acknowledgments During my years in Stanford my technical writing skills developed significantly, unfor- tunately, the same cannot be said for more sentimental genres. Below is my unworthy attempt to acknowledge and thank those who have made my time at Stanford possible, rewarding, exciting, and fun. First and foremost I would like to thank my adviser Bill Dally. Bill’s tremendous support and encouragement were only exceeded by the inspiration and opportunities he gave me. Bill facilitated my studies at Stanford from even before I arrived, let me help plan and teach two new classes, and always gave me the freedom to pursue both my research and extra curricular interests while (mostly) keeping me focused on my thesis. Most importantly, Bill gave me the chance to be deeply involved in the broad and collaborative Merrimac project. I was very lucky that I was able to collaborate and interact with several of Stanford’s remarkable faculty members as part of my thesis and the Merrimac project. I particularly want to thank Pat Hanrahan and Mendel Rosenblum for the continuous feedback they have provided on my research and the final thoughtful comments on this dissertation. Pat and Mendel, as well as Eric Darve and Alex Aiken, taught me much and always offered fresh perspectives and insight that broadened and deepened my knowledge. I want to give special thanks to Ronny Ronen, Uri Weiser, and Adi Yoaz. They gave me a great start in my career in computer architecture and provided careful and extensive guidance and help over the years. This dissertation represents a small part of the effort that was put into the Merrimac project and I would like to acknowledge the amazing students and researchers with whom I collaborated. The Merrimac team included Jung Ho Ahn, Nuwan Jayasena, Abhishek Das, Timothy J. Knight, Francois Labonte, Jayanth Gummaraju, Ben Serebrin, Ujval Kapasi, Binu Mathew, and Ian Buck. I want to particularly thank Jung Ho Ahn, Jayanth vi Gummaraju , and Abhishek Das, with whom I worked most closely, for their thoughts and relentless infrastructure support. Timothy J. Barth, Massimiliano Fatica, Frank Ham, and Alan Wray of the Center for Integrated Turbulence Research provided priceless expertise and showed infinite patience while developing applications for Merrimac. In my time at Stanford, I had the pleasure to work with many other wonderful fellow students outside the scope of my dissertation research. I especially want to thank the members of the CVA group and those of the Sequoia project. Recently, I had the pleasure to participate in the ambitious efforts of Kayvon Fatahlian, Timothy J. Knight, Mike Houston, Jiyoung Park, Manman Ren, Larkhoon Lim, and Abhishek Das on the Sequoia project. Earlier, I spent many rewarding and fun hours with Sarah Harris and Kelly Shaw, learned much about streaming from Ujval Kapasi, Brucek Khailany, John Owens, Peter Mattson, and Scott Rixner, and got a taste of a wide range of other research topics from Brian Towles, Patrick Chiang, Andrew Chang, Arjun Singh, Amit Gupta, Paul Hartke, John Kim, Li-Shiuan Peh, Edward Lee, David Black-Schaffer, Oren Kerem, and James Balfour. I am grateful to the assistance I got over the years from Shelley Russell, Pamela Elliott, Wanda Washington, and Jane Klickman, and to the funding agencies that supported my graduate career at Stanford, including the Department of Energy (under the ASCI Alliances Program, contract LLNL-B523583), the Defense Advanced Research Projects Agency (under the Polymorphous Computing Architectures Project, contract F29601-00- 2-0085), and the Nvidia Fellowship Program. In addition to the friends I met through research, I also want to deeply thank the rest of my friends. Those in Israel who made me feel like I I had never left home whenever I went for visit, and those at Stanford who have gave me a new home and sustained me with a slew of rituals (Thursday parties, afternoon coffee, Sunday basketball, movie nights, ...): Wariya Keatpadungkul, Assaf Yeheskelly, Eilon Brenner, Zvi Kons, Dror Levin, Ido Milstein, Galia Arad-Blum, Eran Barzilai, Shahar Wilner, Dana Porat, Luis Sentis, Vincent Vanhoucke, Onn Brandman, Ofer Levi, Ulrich Barnhoefer, Ira Wygant, Uri Lerner, Nurit Jugend, Rachel Kolodny, Saharon Rosset, Eran Segal, Yuval Nov, Matteo Slanina, Cesar Sanchez, Teresa Madrid, Magnus Sandberg, Adela Ben-Yakar, Laurence Melloul, Minja Trjkla, Oren Shneorson, Noam Sobel, Guy Peled, Bay Thongthepairot, Mor Naaman, Tonya Putnam, Relly Brandman, Calist Friedman, Miki Lustig, Yonit Lustig, Vivian Ganitsky, and Joyce Noah-Vanhoucke. vii Above all I thank my family without whom my years at Stanford would have been both impossible and meaningless. my parents, Lipa and Mia Erez, and my sister, Mor Peleg, gave me a wonderful education, set me off on the academic path, encouraged me in all my choices and made my endeavors possible and simple. My brother in law, Alex Peleg, who introduced me to the field of computer architecture and my nephew and nieces, Tal, Adi, and Yael Peleg, whom I followed to Stanford. My love for them has without a doubt been the single most critical contributor to my success. viii Contents Abstract v Acknowledgments vi 1 Introduction 1 1.1 Contributions ................................... 2 1.2 Detailed Thesis Roadmap and Summary .................... 3 1.2.1 Background ................................ 3 1.2.2 Merrimac Architecture .......................... 3 1.2.3 Streaming Scientific Applications .................... 5 1.2.4 Unstructured and Irregular Algorithms ................ 5 1.2.5 Fault Tolerance .............................. 6 1.2.6 Conclusions ................................ 6 2 Background 7 2.1 Scientific Applications .............................. 7 2.1.1 Numerical Methods ........................... 8 2.1.2 Parallelism ................................ 10 2.1.3 Locality and Arithmetic Intensity .................... 11 2.1.4 Control .................................. 12 2.1.5 Data Access ................................ 13 2.1.6 Scientific Computing Usage Model ................... 13 2.2 VLSI Technology ................................. 13 2.2.1 Logic Scaling ............................... 14 2.2.2 Bandwidth Scaling ............................ 14 ix 2.2.3 Latency scaling .............................. 15 2.2.4 VLSI Reliability ............................. 16 2.2.5 Summary ................................. 16 2.3 Trends in Supercomputers ............................ 17 2.4 Models and Architectures ............................ 19 2.4.1 The von Neumann