1. Application Benchmarking
Total Page:16
File Type:pdf, Size:1020Kb
INTERNATIONAL ROADMAP FOR DEVICES AND SYSTEMS 2020 EDITION APPLICATION BENCHMARKING THE IRDS IS DEVISED AND INTENDED FOR TECHNOLOGY ASSESSMENT ONLY AND IS WITHOUT REGARD TO ANY COMMERCIAL CONSIDERATIONS PERTAINING TO INDIVIDUAL PRODUCTS OR EQUIPMENT. ii © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Wi-Fi® and Wi-Fi Alliance® are registered trademarks of Wi-Fi Alliance. The IEEE emblem is a trademark owned by the IEEE. "IEEE", the IEEE logo, and other IEEE logos and titles (IRDSTM, IEEE 802.11™, IEEE P1785™, IEEE P287™, IEEE P1770™, IEEE P149™, IEEE 1720™, etc.) are registered trademarks or service marks of The Institute of Electrical and Electronics Engineers, Incorporated. All other products, company names or other marks appearing on these sites are the trademarks of their respective owners. Nothing contained in these sites should be construed as granting, by implication, estoppel, or otherwise, any license or right to use any trademark displayed on these sites without prior written permission of IEEE or other trademark owners. NVIDIA is a registered trademark of NVIDIA Corporation in the U.S. and other countries. TESLA is a trademark of TESLA, INC. Google and YouTube are trademarks of Google LLC. are registered trademarks of Google LLC. Microsoft® is a registered trademarks of Microsoft Corporation Amazon is a trademark of Amazon.com, Inc. Facebook is a registered trademark of Facebook, Inc. Netflix is a registered trademark of Netflix, Inc. UBER is a trademark of UBER TECHNOLOGIES, INC. Intel® is a registered trademark of Intel Corporation. Arm® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere. Synopsys® and Arc® are registered trademarks of Synopsys, Inc. Other company and product names may be trademarks of the respective companies with which they are associated. THE INTERNATIONAL TECHNOLOGY ROADMAP FOR SEMICONDUCTORS: 2011 iii Table of Contents Acknowledgments ................................................................................................................. v 1. Introduction ................................................................................................................. 1 1.1. Current State of Technology ........................................................................................... 2 1.2. Drivers and Technology Targets ..................................................................................... 2 1.3. Vision of Future Technology ........................................................................................... 2 2. Scope of Report .......................................................................................................... 2 3. Summary and Key Points ........................................................................................... 3 4. Application Analysis .................................................................................................... 3 4.1. Big Data Analytics........................................................................................................... 3 4.2. Artificial Intelligence ........................................................................................................ 7 4.3. Discrete Event Simulation ............................................................................................. 18 4.4. Physical System Simulation .......................................................................................... 22 4.5. Optimization .................................................................................................................. 27 4.6. Graphics/VR/AR ........................................................................................................... 32 4.7. Cryptographic Codec .................................................................................................... 34 5. Cross Matrix ............................................................................................................. 35 6. Conclusions and Recommendations ........................................................................ 36 7. References ............................................................................................................... 37 THE INTERNATIONAL ROADMAP FOR DEVICES AND SYSTEMS: 2020 COPYRIGHT © 2020 IEEE. ALL RIGHTS RESERVED. iv List of Figures Figure AB-1 Number of GTEPS of the Top Six Machines Since the Inception of the Graph 500 Competition .......................................................................................................... 5 Figure AB-2 Number of Cores Used in the Top Six Graph 500 Machines Since the Inception of the Graph 500 Competition ................................................................................... 6 Figure AB-3 The variety of software and hardware options at various levels greatly complicate the benchmarking of AI/ML systems []. ................................................................. 8 Figure AB-4 General tradeoffs among Computing Architectures .............................................10 Figure AB-5 Compute Efficiency (TOPS/W) of Existing and Projected Hardware for Training .15 Figure AB-6 Data from Figure AB-5 for Existing and Projected Hardware for Training ............16 Figure AB-7 Compute Efficiency of Existing and Projected Hardware for Inference .................17 Figure AB-8 Data from Figure AB-7 for Existing and Projected Hardware for Inference ..........17 Figure AB-9 Efficiency as a function of claimed peak TOPS, showing the percentage of claimed compute power actually delivered in practice. ......................................................18 Figure AB-10 Historical Performance of 471.omnetpp Over Time .............................................20 Figure AB-11 Historical Performance of 620.omnetpp_s over time ............................................20 Figure AB-12 Working Set Analysis of 471.omnetpp and Dramatic Jump in Peak Performance Due to Working Set Fitting in Cache ....................................................................21 Figure AB-13 HPL (dense systems, blue) vs. HPCG (sparse systems, red) Average Performance ........................................................................................................24 Figure AB-14 HPL (dense systems, blue) vs. HPCG (sparse systems, red) Fraction of Peak Performance ........................................................................................................25 Figure AB-15 HPL (dense systems) vs. HPCG (sparse systems) vs. the most energy-efficient supercomputers on the Green 500 list .................................................................26 Figure AB-16 Historical Performance of 473.astar Over Time ...................................................29 Figure AB-17 Working Set Analysis of 473.astar is Approximately 16–20 MB ...........................30 Figure AB-18 Top-down analysis of SPEC CPU 2017 benchmarks on an Intel Core i7-8700K- based system ......................................................................................................31 Figure AB-19 Performance trends of optimization workloads from SPEC CPU 2017 .................31 Figure AB-20 Combined trend of optimization represented by the geomean of 631.deepsjeng_s, 648.exchange2_s, and 641.leela_s .....................................................................32 Figure AB-21 Geometric Mean Across Viewsets vs. Time for SPECviewperf12 and SPECview13 Benchmarks ........................................................................................................33 List of Tables Table AB-1 Application Areas.................................................................................................. 1 Table AB-2 Critical Technology Needs for Application Areas .................................................. 3 Table AB-3 Kernel Problem Classes (the benchmark gives different problem classes, based on the approximate size of the graph. The classes are shown below) ....................... 4 Table AB-4 AI/ML Deployment Platforms and Constraints ...................................................... 9 Table AB-5 AI/ML Benchmarking Metrics ...............................................................................13 Table AB-6 L2 and LLC access breakdown for SPEC Int Speed 2017 ...................................22 Table AB-7 Cross Matrix of Application Area vs. Market Drivers ............................................35 THE INTERNATIONAL ROADMAP FOR DEVICES AND SYSTEMS: 2020 COPYRIGHT © 2020 IEEE. ALL RIGHTS RESERVED. v ACKNOWLEDGMENTS Thanks goes to the members of the team, who are: Name Representing Geoffrey Burr IBM Tom Conte [chair] Georgia Institute of Technology, USA Paolo Gargini Chair, IRDS Vladimir Getov University of Westminster, UK Yoshihiro Hayashi Keio University, JAPAN [SDRJ representative] Takeshi Iwashita* Hokkaido University, JAPAN [SDRJ representative] Vijay Janapa Reddi* Harvard University, USA Masaaki Kondo* University of Tokyo, JAPAN [SDRJ representative] Tushar Krishna* Georgia Institute of Technology, USA Peter M. Kogge University of Notre Dame, USA Anoop Niar* Flextronics, Inc. Dam Sunwoo Arm Research Josep Torrellas University of Illinois at Urbana-Champaign, USA Peter Torelli Chair, EEMBC