OFFICIAL USE ONLY SANDIA REPORT SAND2014-18566 Official Use Only Printed October 2014 Beyond Moore’s Law Computer Architecture Erik P. DeBenedictis, Brad Aimone, Peter Sharma, with additional participants acknowledged inside Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000. OFFICIAL USE ONLY May be exempt from public release under the Freedom of Information Act (5 U.S.C. 552), exemption number and category: 4 Commercial/proprietary Department of Energy review required before public release. Name/Org: Erik P. DeBenedictis Date: October 1, 2014 Guidance (if applicable): Proprietary information of another company for exemption 4. DOES NOT CONTAIN OFFICIAL USE ONLY INFORMATION Name/Org: Erik P. DeBenedictis/1425 Date: 1/17/2016 Further dissemination only as authorized to U.S. Government agencies and their contractors; other requests shall be approved by the originating facility or higher DOE programmatic authority. OFFICIAL USE ONLY 1/96 OFFICIAL USE ONLY Issued by Sandia National Laboratories, operated for the United States Department of Energy by Sandia Corporation. NOTICE: This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government, nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors, or their employees, make any warranty, express or implied, or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represent that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government, any agency thereof, or any of their contractors or subcontractors. The views and opinions expressed herein do not necessarily state or reflect those of the United States Government, any agency thereof, or any of their contractors. OFFICIAL USE ONLY 2/96 OFFICIAL USE ONLY SAND2014-18566 Official Use Only Printed October, 2014 Beyond Moore’s Law Computer Architecture Erik P. DeBenedictis, Brad Aimone, Peter Sharma, with additional participants acknowledged inside Sandia National Laboratories P.O. Box 5800 Albuquerque, New Mexico 87185-MS 1319 Abstract Final report for Beyond Moore’s Law Computer Architecture LDRD 171060. It reports on the development of a computer scaling and computer architecture framework for computers that will apply once electronic devices stop scaling due to maturation of Moore’s Law. The project also includes subprojects in materials and devices, which are either reported here or are projects in alliance with this project Further dissemination only as authorized to U.S. Government agencies and their contractors; other requests shall be approved by the originating facility or higher DOE programmatic authority. OFFICIAL USE ONLY 3/96 OFFICIAL USE ONLY Acknowledgements The following people contributed to this project John Aidun David Frank Peter Mattern Brad Aimone Mike Frank Setso Metodi Jim Ang Scott Hemmert Nancy Missert Ed Barsis Bruce Hendrickson John Naegle Bob Benner David Henry Murat Okandan Geoff Brennecka Vince Hitalia Ben Payne Bill Camp Scott Holmes Steve Plimpton Frances Chance John Ihlefeld Andrew Pomerene Patrick Chu Conrad James Fred Rothganger Jeanine Cook Jim Laros Peter Sharma Rich Dondero François Léonard Ann Speed Tim Draelos Rupert Lewis John Sullivan Bruce Draper Denis Mamaluy Alec Talin Serena Elay Marc Manheimer John Wagner Brian Evans Matt Marinella Noel Wheeler OFFICIAL USE ONLY 4/96 OFFICIAL USE ONLY Table of contents Beyond Moore’s Law Computer Architecture.............................................................. 9 Background ................................................................................................................... 9 Current crisis in computers................................................................................... 10 Sandia Beyond Moore activities and this LDRD ............................................... 10 Project organization ................................................................................................... 11 “Beyond Moore’s Law” Computing Concepts ........................................................ 12 Project status .............................................................................................................. 14 Subprojects in materials and device research................................................... 14 Subprojects in computation research activities ................................................. 14 References .................................................................................................................. 15 Publications............................................................................................................. 15 Documents in preparation..................................................................................... 16 Presentations .......................................................................................................... 16 Workshops............................................................................................................... 16 Appendix I: A Computing Paradigm Beyond Moore’s Law and Beyond the von Neumann Architecture................................................................................................... 17 Abstract.................................................................................................................... 19 Introduction.................................................................................................................. 19 Energy efficiency of logic families............................................................................ 21 Optimal clock rate....................................................................................................... 22 Optimal Adiabatic Scaling......................................................................................... 23 3D manufacture and distant future vision............................................................... 24 Progression over time................................................................................................ 25 Synergy of computer architecture and Optimal Adiabatic Scaling ..................... 26 Processor-In-Memory-and-Storage (PIMS) implementation............................... 29 Example of sparse matrices for neurons and meshes......................................... 31 Directional transpose matrix representation ...................................................... 31 Sparse matrices...................................................................................................... 33 ALU complexity....................................................................................................... 34 Power consumption ............................................................................................... 35 Memory Access Energy ........................................................................................ 37 Compute Energy..................................................................................................... 38 System Energy ....................................................................................................... 39 Adiabatic circuit families............................................................................................ 40 Conclusions................................................................................................................. 42 References .................................................................................................................. 43 Appendix II: Artificial Neural Networks as Beyond CMOS Systems...................... 47 Abstract.................................................................................................................... 49 Introduction.................................................................................................................. 49 Reasoning about the limits of technology .............................................................. 50 Landauer’s principle and minimum gate energy ............................................... 52 Communications limits .......................................................................................... 53 Computational complexity theory and algorithms ............................................. 53 Proposed method................................................................................................... 54 Utility of the method ............................................................................................... 55 OFFICIAL USE ONLY 5/96 OFFICIAL USE ONLY Analysis of a level-based ANN............................................................................. 56 A neural-network algorithm to improve energy efficiency.................................... 61 Extending the hierarchy of neural-network algorithms......................................... 63 Digital computer implementations ..........................................................................