DOCSLIB.ORG

World's First High- Performance X86 With

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
World's First High- Performance X86 With World’s First High- Performance x86 with Integrated AI Coprocessor Linley Spring Processor Conference 2020 April 8, 2020 G Glenn Henry Dr. Parviz Palangpour Chief AI Architect AI Software Deep Dive into Centaur’s New x86 AI Coprocessor (Ncore) • Background • Motivations • Constraints • Architecture • Software • Benchmarks • Conclusion Demonstrated Working Silicon For Video-Analytics Edge Server Nov, 2019 ©2020 Centaur Technology. All Rights Reserved Centaur Technology Background • 25-year-old startup in Austin, owned by Via Technologies • We design, from scratch, low-cost x86 processors • Everything to produce a custom x86 SoC with ~100 people • Architecture, logic design and microcode • Design, verification, and layout • Physical build, fab interface, and tape-out • Shipped by IBM, HP, Dell, Samsung, Lenovo… ©2020 Centaur Technology. All Rights Reserved Genesis of the AI Coprocessor (Ncore) • Centaur was developing SoC (CHA) with new x86 cores • Targeted at edge/cloud server market (high-end x86 features) • Huge inference markets beyond hyperscale cloud, IOT and mobile • Video analytics, edge computing, on-premise servers • However, x86 isn’t efficient at inference • High-performance inference requires external accelerator • CHA has 44x PCIe to support GPUs, etc. • But adds cost, power, another point of failure, etc. ©2020 Centaur Technology. All Rights Reserved Why not integrate a coprocessor? • Very low cost • Many components already on SoC (“free” to Ncore) Caches, memory, clock, power, package, pins, busses, etc. • There often is “free” space on complex SOCs due to I/O & pins • Having many high-performance x86 cores allows flexibility • The x86 cores can do some of the work, in parallel • Didn’t have to implement all strange/new functions • Allows fast prototyping of new things • For customer: nothing extra to buy ©2020 Centaur Technology. All Rights Reserved Coprocessor Challenges • Have to live with the constraints of the rest of the chip • Technology choice, clock speeds, power distribution, memory, etc. • Especially size and aspect ratio of available “hole” that may change during development • There is no good coprocessor architecture for x86 • We had to invent one. Will it work? Will it last? • Who is going to work on it? • Everyone already busy working on x86 cores and the rest of the SoC • RTL, physical design, verification, software must be done with very small team ©2020 Centaur Technology. All Rights Reserved Our Original Objectives/Focus • Target application is edge-server inference • Support applications without retraining • Best performance/total system $ • For systems with a “significant” level of performance (>1000 ResNet50) • Secondary goal is lowest latency – important for edge server • Key internal design metric: MAC efficiency with limited number of MACs • Will NOT have: • Best performance/W (attached to a processor & optimized for performance) • Best raw performance (limited by CHA die-size objectives) ©2020 Centaur Technology. All Rights Reserved CHA Chip Structure misc I/O 44 PCI lanes • Eight x86 cores Southbridge PCIE I/O functions • Similar IPC as Intel Haswell • Plus AVX512 instructions • Totally new core vs our previous x86 core L3 L3 x86 core • 16MB L3 cache x86 core L3 L3 x86 core • 2x512-bit ring interconnects x86 core L3 L3 x86 core • 4 DDR4 controllers • Lots of PCIe lanes x86 core L3 L3 x86 core • Multi-socket support 4x DDR4 DRAM controllers • Running at 2.5 GHz ©2020 Centaur Technology. All Rights Reserved CHA Chip Structure misc I/O 44 PCI lanes Southbridge PCIE I/O functions Ncore Ncore is attached to the ring x86 core L3 L3 x86 core x86 core L3 L3 x86 core Thus, it can easily communicate with cores, L3, x86 core x86 core L3 L3 DRAM & I/O x86 core L3 L3 x86 core 4x DDR4 DRAM controllers ©2020 Centaur Technology. All Rights Reserved I/O & PCI (44 lanes) CHA NCORE Interconnect rings RAM bus & debug socket dual 2 194 mm 4 4 in TMSC cores cores 16nm FFC NCORE COMPUTE 16MB L3 NCORE RAM 4x DDR controllers DRAM controllers Ncore DMA requests, tags & wait on tags Connectivity load/store 64B/clk to/from x86 core to Ncore PCI & I/O Two DMA engines (read,write) L3 L3L3 Ring 3 access via virtual address given Memory space allocated by cachecachecache device driver to process at open() L2 rd/wr controlled by Ncore . L2L2 . .. L1L1L1 intr PCI config corecorecore IPI or APIC new 64b x86 streaming insts to speed ncore read/write slave (to core) Interface (optional, can also use AVX512 mov) RAM status, control, compute instructions, etc. ©2020 Centaur Technology. All Rights Reserved Ncore Basic Architecture • Remember: efficient, scalable and flexible (and low latency) • Systolic array of MAC not very attractive • Hard to do (fast) non-MAC things • Can we achieve high utilization with our limited area? • Instead, we choose a very wide SIMD architecture organized into vertical “slices” • Very efficient wrt. area • Easy to subdivide into vertical slices for scalability • Allows easy add of non-MAC functions (like data functions) • Very good latency (all accumulators available in 1 clock) • We know how to make SIMD fast (GHz) and area efficient • Run as fast (GHz) as possible since it directly contributes to performance – but not faster than cores ©2020 Centaur Technology. All Rights Reserved 16 slices The x Final 256 bytes wide Ncore RAM = 4,096 bytes wide logic “AVX32768” Structure inst_memory Instruction All running at Buses sequencer 2.5 GHz 20 Teraops/sec 2,048 RAM “rows” x 4KB/row x 2 RAMs = 16MB of RAM ©2020 Centaur Technology. All Rights Reserved External buses (each 512-bits) Instruction ROM Bus Instruction memory zero-clock Ncore subsystem bus control branches Instruction decoder & sequencer Architecture RAM arbitration RAM controls D-RAM W-RAM 4096 byte-wide pipeline NDU Neural Data Unit controls Running at 2.5GHz NPU controls NPU accumulators OUT-unit OUT-unit controls ©2020 Centaur Technology. All Rights Reserved External buses (each 512-bits) Instruction ROM 256 insts Instruction Bus memory 2x256 insts Ncore subsystem bus control (banked) Instruction decoder & sequencer Architecture RAM arbitration One instruction controls entire pipeline Instructions are 128-bit low-level ucode-like RAM controls D-RAM W-RAM All instructions execute in one clock (incl. 0-clock branches, loops, call/ret, etc.) Each pipeline stage executes in one clock NDU Neural Data Unit Instructions can also control DMA controls 4096 byte-wide pipeline NPU controls NPU accumulators Except… some OUT functions are muxed down (pipelined) 8:1 but pipeline OUT-unit OUT-unit controls above can continue to run ©2020 Centaur Technology. All Rights Reserved Bus subsystem Neural Data Unit is attached to the The side-slice NDUs such that it can rotate an entire 4096-byte “row” in 1 clock Execution NDU functions have 8 possible inputs & 4 output registers (all 4,096B) Pipeline D-RAM W-RAM NDU performs 3 parallel functions from: RAM outputs -- Pass input to output reg(s) -- “Compress” blocks (for pooling) -- Rotate entire row (4,096B) NDU by 1, 2, 4, 8,16, 32, 64, -1 bytes -- “Broadcast” bytes (expand weights) MOV outputs -- “Edge swap” -- Merge input with NDU output reg using NPU another NDU reg as byte mask -- etc. accumulators All functions execute in one clock OUT-unit out data Bus subsystem The NPU supports 9b & 16b integer & bfloat16 Execution --9b integer MAC takes 1 clk --bfloat16 MAC takes 3 clks Pipeline D-RAM W-RAM Functions = MACs, adds, subtracts, RAM outputs min/max, logic, etc., lots of variations Each NPU has 8 predication registers & ops to conditionally update accumulator NDU Saturating 32b accumulator MOV outputs (32b integer or single-precision FP) Inputs come from NDU registers NPU & forwarded NPU inputs accumulators Built-in 8b→9b zero-quantizations (subtract offsets) OUT-unit out data Bus subsystem The OUT unit performs quantization Execution of 32b accumulator to 8b,16b & bfloat16, Pipeline D-RAM W-RAM Quantization based on Google, PyTorch, etc. RAM outputs Also does ReLU, tanh & sigmoid Results go into 2 “out” registers, that can be stored directly to RAM NDU or forwarded to NDU input MOV outputs Full 32b accumulator can also be stored “horizontally” or “vertically” NPU All functions execute in 1 or 3 clocks, accumulators but some functions are pipelined muxed 8:1 (≈10 clocks worst case OUT) OUT-unit out data Bus subsystem The Execution Pipeline D-RAM W-RAM RAM outputs NDU MOV outputs NPU OUTs are relatively rare accumulators But, the pipeline above (more MACs, etc) can continue to run overlapped with OUT OUT-unit out data Ncore Implementation Challenges • 32,768-bit datapath ! • Connecting data & controls must be fanned-out/stages many clocks ahead • Multiple clocks & thousands of clock enables • Datapath is not always regular (different bytes can have different logic) • Etc → lots of RTL just managing connections • Critical “loop” stage timing (2.5 GHz) • Sequencer… only ~150ps left to do anything (after inst RAM fetch) • The NPU = MAC + lots of other functions, predication, saturation… • The NDU = massive wiring and rotate connections • Large, tightly coupled 4KB-wide RAM (w/ECC) • Timing, managing arbitration • Getting data to/from 4 write sources and 4 read sinks • Tight interaction between RTL & physical build is critical ! • Also need a really good build person (we have one) ©2020 Centaur Technology. All Rights Reserved Non- NDU RAM Portion NPU of Ncore & OUT seq I/O processing 2.1K I/O ~11mm2 in 16nm ~1.3M regs & 19.5M gates Ncore Machine Instructions • Instruction = 128b of fields controlling various pieces of pipeline • For instance, a single instruction could have: 30b: control of 2 RAM reads & index operations 22b: branch control 20b: control of NPU 15b: control of NDU write to RAM 26b: control of NDU 15b: misc • Only about ½ – ¾ of instruction bits used on most instructions • But some important functions use more • Several hardware considerations dictate the 128b size • Hardware dependent → will change with new hardware version • Also, understanding of hardware required for efficient programming • Thus, detailed instruction definition will not be made public • But, nobody needs to know because we’ve got SW tools & a stack… ©2020 Centaur Technology.
Load more

Recommended publications
  • X86 Platform Coprocessor/Prpmc (PC on a PMC)
    x86 Platform Coprocessor/PrPMC (PC on a PMC) 32b/33MHz PCI bus PN1/PN2 +5V to Kbd/Mouse/USB power Vcore Power +3.3V +2.5v Conditioning +3.3VIO 1K100 FPGA & 512KB 10/100TX TMS2250 PCI BIOS/PLA Ethernet RJ45 Compact to PCI bridge Flash Memory PLA I/O Flash site 8 32b/33MHz Internal PCI bus Analog SVGA Video Pwr Seq AMD SC2200 Signal COM 1 (RXD/TXD only) IDE "GEODE (tm)" Conditioning COM 2 (RXD/TXD only) Processor USB Port 1 Rear I/O PN4 I/O Rear 64 USB Port 2 LPC Keyboard/Mouse Floppy 36 Pin 36 Pin Connector PC87364 128MB SDRAM Status LEDs Super I/O (16MWx64b) PC Spkr This board is a Processor PMC (PrPMC) implementing A PrPMC implements a TMS2250 PCI-to-PCI bridge to an x86-based PC architecture on a single-wide, stan- the host, and a “Coprocessor” cofniguration implements dard height PMC form factor. This delivers a complete back-to-back 16550 compatible COM ports. The “Mon- x86 platform with comprehensive I/O support, a 10/100- arch” signal selects either PrPMC or Co-processor TX Ethernet interface, and disk access (both floppy and mode. IDE drives). The board features an on-board hard drive using Compact Flash. A 512Kbyte FLASH memory holds the 1K100 PLA im- age that is automatically loaded on power up. It also At the heart of the design is an Advanced Micro De- contains the General Software Embedded BIOS 2000™ vices SC2200 GEODE™ processor that integrates video, BIOS code that is included with each board.
    [Show full text]
  • Convey Overview
    THE WORLD’S FIRST HYBRID-CORE COMPUTER. CONVEY HYBRID-CORE COMPUTING Hybrid-core Computing Convey HC-1 High Performance of application- specific hardware Heterogenous solutions • can be much more efficient Performance/ • still hard to program Programmability and Power efficiency deployment ease of an x86 server Application Multicore solutions • don’t always scale well • parallel programming is hard Low Difficult Ease of Deployment Easy 1/22/2010 3 Hybrid-Core Computing Application-Specific Personalities Applications • Extend the x86 instruction set • Implement key operations in Convey Compilers hardware Life Sciences x86-64 ISA Custom ISA CAE Custom Financial Oil & Gas Shared Virtual Memory Cache-coherent, shared memory • Both ISAs address common memory *ISA: Instruction Set Architecture 7/12/2010 4 HC-1 Hardware PCI I/O FPGA FPGA Intel Personalities Chipset FPGA FPGA 8 GB/s 80 GB/s Memory Memory Cache Coherent, Shared Virtual Memory 1/22/2010 5 Using Personalities C/C++ Fortran • Personalities are user specifies reloadable personality at instruction sets Convey Software compile time Development Suite • Compiler instruction generates x86 descriptions and coprocessor instructions from Hybrid-Core Executable P ANSI standard x86-64 and Coprocessor Personalities C/C++ & Fortran Instructions • Executable can run on x86 nodes FPGA Convey HC-1 or Convey Hybrid- bitfiles Core nodes Intel x86 Coprocessor personality loaded at runtime by OS 1/22/2010 6 SYSTEM ARCHITECTURE HC-1 Architecture “Commodity” Intel Server Convey FPGA-based coprocessor Direct
    [Show full text]
  • Exploiting Free Silicon for Energy-Efficient Computing Directly
    Exploiting Free Silicon for Energy-Efficient Computing Directly in NAND Flash-based Solid-State Storage Systems Peng Li Kevin Gomez David J. Lilja Seagate Technology Seagate Technology University of Minnesota, Twin Cities Shakopee, MN, 55379 Shakopee, MN, 55379 Minneapolis, MN, 55455 [email protected] [email protected] [email protected] Abstract—Energy consumption is a fundamental issue in today’s data A well-known solution to the memory wall issue is moving centers as data continue growing dramatically. How to process these computing closer to the data. For example, Gokhale et al [2] data in an energy-efficient way becomes more and more important. proposed a processor-in-memory (PIM) chip by adding a processor Prior work had proposed several methods to build an energy-efficient system. The basic idea is to attack the memory wall issue (i.e., the into the main memory for computing. Riedel et al [9] proposed performance gap between CPUs and main memory) by moving com- an active disk by using the processor inside the hard disk drive puting closer to the data. However, these methods have not been widely (HDD) for computing. With the evolution of other non-volatile adopted due to high cost and limited performance improvements. In memories (NVMs), such as phase-change memory (PCM) and spin- this paper, we propose the storage processing unit (SPU) which adds computing power into NAND flash memories at standard solid-state transfer torque (STT)-RAM, researchers also proposed to use these drive (SSD) cost. By pre-processing the data using the SPU, the data NVMs as the main memory for data-intensive applications [10] to that needs to be transferred to host CPUs for further processing improve the system energy-efficiency.
    [Show full text]
  • CUDA What Is GPGPU
    What is GPGPU ? • General Purpose computation using GPU in applications other than 3D graphics CUDA – GPU accelerates critical path of application • Data parallel algorithms leverage GPU attributes – Large data arrays, streaming throughput Slides by David Kirk – Fine-grain SIMD parallelism – Low-latency floating point (FP) computation • Applications – see //GPGPU.org – Game effects (FX) physics, image processing – Physical modeling, computational engineering, matrix algebra, convolution, correlation, sorting Previous GPGPU Constraints CUDA • Dealing with graphics API per thread per Shader Input Registers • “Compute Unified Device Architecture” – Working with the corner cases per Context • General purpose programming model of the graphics API Fragment Program Texture – User kicks off batches of threads on the GPU • Addressing modes Constants – GPU = dedicated super-threaded, massively data parallel co-processor – Limited texture size/dimension Temp Registers • Targeted software stack – Compute oriented drivers, language, and tools Output Registers • Shader capabilities • Driver for loading computation programs into GPU FB Memory – Limited outputs – Standalone Driver - Optimized for computation • Instruction sets – Interface designed for compute - graphics free API – Data sharing with OpenGL buffer objects – Lack of Integer & bit ops – Guaranteed maximum download & readback speeds • Communication limited – Explicit GPU memory management – Between pixels – Scatter a[i] = p 1 Parallel Computing on a GPU Extended C • Declspecs • NVIDIA GPU Computing Architecture – global, device, shared, __device__ float filter[N]; – Via a separate HW interface local, constant __global__ void convolve (float *image) { – In laptops, desktops, workstations, servers GeForce 8800 __shared__ float region[M]; ... • Keywords • 8-series GPUs deliver 50 to 200 GFLOPS region[threadIdx] = image[i]; on compiled parallel C applications – threadIdx, blockIdx • Intrinsics __syncthreads() – __syncthreads ..
    [Show full text]
  • Comparing the Power and Performance of Intel's SCC to State
    Comparing the Power and Performance of Intel’s SCC to State-of-the-Art CPUs and GPUs Ehsan Totoni, Babak Behzad, Swapnil Ghike, Josep Torrellas Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA E-mail: ftotoni2, bbehza2, ghike2, [email protected] Abstract—Power dissipation and energy consumption are be- A key architectural challenge now is how to support in- coming increasingly important architectural design constraints in creasing parallelism and scale performance, while being power different types of computers, from embedded systems to large- and energy efficient. There are multiple options on the table, scale supercomputers. To continue the scaling of performance, it is essential that we build parallel processor chips that make the namely “heavy-weight” multi-cores (such as general purpose best use of exponentially increasing numbers of transistors within processors), “light-weight” many-cores (such as Intel’s Single- the power and energy budgets. Intel SCC is an appealing option Chip Cloud Computer (SCC) [1]), low-power processors (such for future many-core architectures. In this paper, we use various as embedded processors), and SIMD-like highly-parallel archi- scalable applications to quantitatively compare and analyze tectures (such as General-Purpose Graphics Processing Units the performance, power consumption and energy efficiency of different cutting-edge platforms that differ in architectural build. (GPGPUs)). These platforms include the Intel Single-Chip Cloud Computer The Intel SCC [1] is a research chip made by Intel Labs (SCC) many-core, the Intel Core i7 general-purpose multi-core, to explore future many-core architectures. It has 48 Pentium the Intel Atom low-power processor, and the Nvidia ION2 (P54C) cores in 24 tiles of two cores each.
    [Show full text]
  • MICROPROCESSOR REPORT the INSIDERS’ GUIDE to MICROPROCESSOR HARDWARE Slot Vs
    VOLUME 12, NUMBER 1 JANUARY 26, 1998 MICROPROCESSOR REPORT THE INSIDERS’ GUIDE TO MICROPROCESSOR HARDWARE Slot vs. Socket Battle Heats Up Intel Prepares for Transition as Competitors Boost Socket 7 A A look Look by Michael Slater ship as many parts as they hoped, especially at the highest backBack clock speeds where profits are much greater. The past year has brought a great deal The shift to 0.25-micron technology will be central to of change to the x86 microprocessor 1998’s CPU developments. Intel began shipping 0.25-micron A market, with Intel, AMD, and Cyrix processors in 3Q97; AMD followed late in 1997, IDT plans to LookA look replacing virtually their entire product join in by mid-98, and Cyrix expects to catch up in 3Q98. Ahead ahead lines with new devices. But despite high The more advanced process technology will cut power con- hopes, AMD and Cyrix struggled in vain for profits. The sumption, allowing sixth-generation CPUs to be used in financial contrast is stark: in 1997, Intel earned a record notebook systems. The smaller die sizes will enable higher $6.9 billion in net profit, while AMD lost $21 million for the production volumes and make it possible to integrate an L2 year and Cyrix lost $6 million in the six months before it was cache on the CPU chip. acquired by National. New entrant IDT added another com- The processors from Intel’s challengers have lagged in petitor to the mix but hasn’t shipped enough products to floating-point and MMX performance, which the vendors become a significant force.
    [Show full text]
  • MPR Article Template
    CENTAUR ADDS AI TO SERVER PROCESSOR First x86 SoC to Integrate Deep-Learning Accelerator By Linley Gwennap (December 2, 2019) ................................................................................................................... Centaur is galloping back into the x86 market with Centaur Technology has designed x86 CPUs and pro- an innovative processor design that combines eight high- cessors for Via for more than 20 years, but we’ve heard little performance CPUs with a custom deep-learning accelerator from the Texan design shop since the debut of the dual-core (DLA). The company is the first to announce a server- Nano X2, which was built in a leading-edge 40nm process processor design that integrates a DLA. The new accelerator, (see MPR 1/24/11, “Via Doubles Down at CES”). Henry, called Ncore, delivers better neural-network performance who managed the company since its inception, recently than even the most powerful Xeon, but without incurring handed the reins to new president Al Loper, another long- the high cost of an external GPU card. The Via Technologies time Centaurian. Henry, boasting 50 years of CPU-design subsidiary began testing the silicon in September; we esti- experience, continues as the company’s AI architect. mate the first products based on this design could enter pro- duction in 2H20, although Via has disclosed no product One Tractor Pulling 4,096 Trailers plans. When designing the Ncore accelerator, Centaur was con- Ncore, which operates as a coprocessor, avoids the cerned that the rapid pace of neural-network evolution vogue MAC array, instead relying on a more traditional pro- grammable SIMD engine. But Centaur took the multiple in 4x DDR4 To other CHA SIMD to the extreme, designing a unit that processes 4,096 bytes in parallel to achieve peak performance of 20 trillion operations per second (TOPS) for 8-bit integers (INT8).
    [Show full text]
  • Introduction to Cpu
    microprocessors and microcontrollers - sadri 1 INTRODUCTION TO CPU Mohammad Sadegh Sadri Session 2 Microprocessor Course Isfahan University of Technology Sep., Oct., 2010 microprocessors and microcontrollers - sadri 2 Agenda • Review of the first session • A tour of silicon world! • Basic definition of CPU • Von Neumann Architecture • Example: Basic ARM7 Architecture • A brief detailed explanation of ARM7 Architecture • Hardvard Architecture • Example: TMS320C25 DSP microprocessors and microcontrollers - sadri 3 Agenda (2) • History of CPUs • 4004 • TMS1000 • 8080 • Z80 • Am2901 • 8051 • PIC16 microprocessors and microcontrollers - sadri 4 Von Neumann Architecture • Same Memory • Program • Data • Single Bus microprocessors and microcontrollers - sadri 5 Sample : ARM7T CPU microprocessors and microcontrollers - sadri 6 Harvard Architecture • Separate memories for program and data microprocessors and microcontrollers - sadri 7 TMS320C25 DSP microprocessors and microcontrollers - sadri 8 Silicon Market Revenue Rank Rank Country of 2009/2008 Company (million Market share 2009 2008 origin changes $ USD) Intel 11 USA 32 410 -4.0% 14.1% Corporation Samsung 22 South Korea 17 496 +3.5% 7.6% Electronics Toshiba 33Semiconduc Japan 10 319 -6.9% 4.5% tors Texas 44 USA 9 617 -12.6% 4.2% Instruments STMicroelec 55 FranceItaly 8 510 -17.6% 3.7% tronics 68Qualcomm USA 6 409 -1.1% 2.8% 79Hynix South Korea 6 246 +3.7% 2.7% 812AMD USA 5 207 -4.6% 2.3% Renesas 96 Japan 5 153 -26.6% 2.2% Technology 10 7 Sony Japan 4 468 -35.7% 1.9% microprocessors and microcontrollers
    [Show full text]
  • AI Chips: What They Are and Why They Matter
    APRIL 2020 AI Chips: What They Are and Why They Matter An AI Chips Reference AUTHORS Saif M. Khan Alexander Mann Table of Contents Introduction and Summary 3 The Laws of Chip Innovation 7 Transistor Shrinkage: Moore’s Law 7 Efficiency and Speed Improvements 8 Increasing Transistor Density Unlocks Improved Designs for Efficiency and Speed 9 Transistor Design is Reaching Fundamental Size Limits 10 The Slowing of Moore’s Law and the Decline of General-Purpose Chips 10 The Economies of Scale of General-Purpose Chips 10 Costs are Increasing Faster than the Semiconductor Market 11 The Semiconductor Industry’s Growth Rate is Unlikely to Increase 14 Chip Improvements as Moore’s Law Slows 15 Transistor Improvements Continue, but are Slowing 16 Improved Transistor Density Enables Specialization 18 The AI Chip Zoo 19 AI Chip Types 20 AI Chip Benchmarks 22 The Value of State-of-the-Art AI Chips 23 The Efficiency of State-of-the-Art AI Chips Translates into Cost-Effectiveness 23 Compute-Intensive AI Algorithms are Bottlenecked by Chip Costs and Speed 26 U.S. and Chinese AI Chips and Implications for National Competitiveness 27 Appendix A: Basics of Semiconductors and Chips 31 Appendix B: How AI Chips Work 33 Parallel Computing 33 Low-Precision Computing 34 Memory Optimization 35 Domain-Specific Languages 36 Appendix C: AI Chip Benchmarking Studies 37 Appendix D: Chip Economics Model 39 Chip Transistor Density, Design Costs, and Energy Costs 40 Foundry, Assembly, Test and Packaging Costs 41 Acknowledgments 44 Center for Security and Emerging Technology | 2 Introduction and Summary Artificial intelligence will play an important role in national and international security in the years to come.
    [Show full text]
  • Instruction Set Innovations for Convey's HC-1 Computer
    Instruction Set Innovations for Convey's HC-1 Computer THE WORLD’S FIRST HYBRID-CORE COMPUTER. Hot Chips Conference 2009 [email protected] Introduction to Convey Computer • Company Status – Second round venture based startup company – Product beta systems are at customer sites – Currently staffing at 36 people – Located in Richardson, Texas • Investors – Four Venture Capital Investors • Interwest Partners (Menlo Park) • CenterPoint Ventures (Dallas) • Rho Ventures (New York) • Braemar Energy Ventures (Boston) – Two Industry Investors • Intel Capital • Xilinx Presentation Outline • Overview of HC-1 Computer • Instruction Set Innovations • Application Examples Page 3 Hot Chips Conference 2009 What is a Hybrid-Core Computer ? A hybrid-core computer improves application performance by combining an x86 processor with hardware that implements application-specific instructions. ANSI Standard Applications C/C++/Fortran Convey Compilers x86 Coprocessor Instructions Instructions Intel® Processor Hybrid-Core Coprocessor Oil & Gas& Oil Financial Sciences Custom CAE Application-Specific Personalities Cache-coherent shared virtual memory Page 4 Hot Chips Conference 2009 What Is a Personality? • A personality is a reloadable set of instructions that augment x86 application the x86 instruction set Processor specific – Applicable to a class of applications instructions or specific to a particular code • Each personality is a set of files that includes: – The bits loaded into the Coprocessor – Information used by the Convey compiler • List of
    [Show full text]
  • CUDA C++ Programming Guide
    CUDA C++ Programming Guide Design Guide PG-02829-001_v11.4 | September 2021 Changes from Version 11.3 ‣ Added Graph Memory Nodes. ‣ Formalized Asynchronous SIMT Programming Model. CUDA C++ Programming Guide PG-02829-001_v11.4 | ii Table of Contents Chapter 1. Introduction........................................................................................................ 1 1.1. The Benefits of Using GPUs.....................................................................................................1 1.2. CUDA®: A General-Purpose Parallel Computing Platform and Programming Model....... 2 1.3. A Scalable Programming Model.............................................................................................. 3 1.4. Document Structure................................................................................................................. 5 Chapter 2. Programming Model.......................................................................................... 7 2.1. Kernels.......................................................................................................................................7 2.2. Thread Hierarchy...................................................................................................................... 8 2.3. Memory Hierarchy...................................................................................................................10 2.4. Heterogeneous Programming................................................................................................11 2.5. Asynchronous
    [Show full text]
  • Intel Xeon Phi Product Family Brief
    PRODUCT BRIEF The Intel® Xeon Phi™ Product Family Highly-Parallel Processing for Unparalleled Discovery Breakthrough Performance for Your Highly-Parallel Applications Extracting extreme performance from highly-parallel “Moving a code to Intel Xeon Phi might applications just got easier. Intel® Xeon Phi™ coprocessors, involve sitting down and adding a based on Intel Many Integrated Core (MIC) architecture, complement the industry-leading performance and couple lines of directives that takes energy-efficiency of the Intel® Xeon® processor E5 family a few minutes. Moving a code to to enable dramatic performance gains for some of today’s most demanding applications—up to 1.2 teraflops a GPU is a project.”3 per coprocessor.1 You can now achieve optimized performance for even your most highly-parallel technical –Dan Stanzione, Deputy Director at computing workloads, while maintaining a unified Texas Advanced Computing Center hardware and software environment.2 The Intel Xeon Phi Coprocessor Even Higher Efficiency for Parallel Processing A Family of Coprocessors for Diverse Needs While a majority of applications will continue to achieve Intel Xeon Phi coprocessors provide up to 61 cores, 244 maximum performance using Intel Xeon processors, threads, and 1.2 teraflops of performance, and they certain highly-parallel applications will benefit dramatically come in a variety of configurations to address diverse by using Intel Xeon Phi coprocessors. Each coprocessor hardware, software, workload, performance, and efficiency features many more and smaller cores, many more threads, requirements.1 They also come in a variety of form factors, and wider vector units. The high degree of parallelism including a standard PCIe* x16 form factor (with active, compensates for the lower speed of each individual core passive, or no thermal solution), and a dense form factor to deliver higher aggregate performance for highly- that offers additional design flexibility (Table 2).
    [Show full text]