DOCSLIB.ORG

GPU: Power Vs Performance

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
GPU: Power Vs Performance IT 14 015 Examensarbete 30 hp June 2014 GPU: Power vs Performance Siddhartha Sankar Mondal Institutionen för informationsteknologi Department of Information Technology Abstract GPU: Power vs Performance Siddhartha Sankar Mondal Teknisk- naturvetenskaplig fakultet UTH-enheten GPUs are widely being used to meet the ever increasing demands of High performance computing. High-end GPUs are one of the highest consumers of power Besöksadress: in a computer. Power dissipation has always been a major concern area for computer Ångströmlaboratoriet Lägerhyddsvägen 1 architects. Due to power efficiency demands modern CPUs have moved towards Hus 4, Plan 0 multicore architectures. GPUs are already massively parallel architectures. There has been some encouraging results for power efficiency in CPUs by applying DVFS . The Postadress: vision is that a similar approach would also provide encouraging results for power Box 536 751 21 Uppsala efficiency in GPUs. Telefon: In this thesis we have analyzed the power and performance characteristics of GPU at 018 – 471 30 03 different frequencies. To help us achieve that, we have made a set of Telefax: microbenchmarks with different levels of memory boundedness and threads. We have 018 – 471 30 00 also used benchmarks from CUDA SDK and Parboil. We have used a GTX580 Fermi based GPU. We have also made a hardware infrastructure that accurately measures Hemsida: the power being consumed by the GPU. http://www.teknat.uu.se/student Handledare: Stefanos Kaxiras Ämnesgranskare: Stefanos Kaxiras Examinator: Ivan Christoff IT 14 015 Sponsor: UPMARC Tryckt av: Reprocentralen ITC Ahëuouu± I would like to thank my supervisor Prof. Stefanos Kaxiras for giving me the wonderful opportunity to work on this interesting topic. I would also like to thank his PhD students Vasileios Spiliopoulos and Konstantinos Koukos for helping me get started with the benchmarks and hardware setup. e LATEX community has been of great help while making this document. Also, I would like to thank the spirit that lives in the computer. is thesis was funded by UPMARC. v C±u±« Ahëuouu± v C±u±« vii ÕI±§o¶h± Õ ó BZh§¶o ì ó.Õ CUDA Programming modelì ó.ó GPU Architecture¦ ó.ì Power issues¢ ó.¦ Latencyä ó.¢ Previous workß ì Mu±oí É ì.Õ Experimental SetupÉ ì.ó Power measurementÉ ì.ì DVFSÕÕ ì.¦ MicrobenchmarksÕÕ ì.¢ Benchmarks Õó ì.¢.Õ Matrix Multiplication Õì ì.¢.ó Matrix Transpose Õì ì.¢.ì Histogram Õì ì.¢.¦ Radix Sort Õì ì.¢.¢ Merge Sort Õì ì.¢.ä Conjugate Gradient Õì ì.¢.ß BFS(Breadth First Search) Õì ì.¢. Eigenvalues Õì ì.¢.É Black-Scholes option pricing Õì ì.¢.Õþ ìD FDTD(ìD Finite Dierence Time Domain method) Õ¦ ì.¢.ÕÕ Scalar Product Õ¦ ¦ EêZ¶Z± Õ¢ vii viii h±u±« ¦.Õ Microbenchmarks Õä ¦.Õ.Õ Microbenchmark Õ Õä ¦.Õ.ó Microbenchmark ó Õß ¦.Õ.ì Microbenchmark ì Õ ¦.Õ.¦ Microbenchmark ¦ ÕÉ ¦.ó Benchmarks óþ ¦.ì Memory Bandwidth ó¦ ¦.ì.Õ Black-Scholes ó¦ ¦.ì.ó Eigenvalue ó¢ ¦.ì.ì ìD FDTD óä ¦.ì.¦ Scalar Product óß ¢ Ch¶« óÉ Bf§Z£í ìÕ A««ufí hou ìì Pëu§ uZ«¶§uu±« ìß 1 hZ£±u§ I±§o¶h± Over the last few years the CPU and GPU architectures have been evolving very rapidly. With the introduction of programmable shaders in GPU since the turn of century, it has been used by the scientic computing community as a powerful computational accelerator to the CPU. For many compute intensive workloads GPUs gives few orders of magnitude better performance than a CPU. e introduction of languages like CUDA and OpenCL has made it more easier to program on a GPU. In our evaluation we have used the Nvidia GTX¢þ GPU. It is based on the Fermi architecture[Õ¢]. With ¢Õó compute units called CUDA cores it can handle ó¦,¢ßä active threads [É, Õ¢]. Its o-chip memory bandwidth is around ÕÉó GB/s, which is considerably higher than CPUs main memory bandwidth(around ìó GB/s for core iß). eoretically it can do over Õ.¢ TFLOPS(with single precision FMA). Even with such great numbers to boast about, GPUs have a few bottlenecks that one has to consider before sending all workload down to the GPU. One of them is the power and the other is the cost to transfer data from host to device(i.e. CPU to GPU) and again back from device to host(i.e. GPU to CPU). e bandwidth of Õäx PCI express is around GB/s. Power is a very important constraint for every computer architecture[ß]. Modern high-end GPUs have thermal design power(TDP) of over óþþ watts. e GTX¢þ has a TDP of ó¦¦ watts. Whereas a high-end multicore CPU like Intel core iß has a TDP of around Õþþ watts. In terms of GFLOPS/watt GPUs can be considered to be more power ecient than CPUs. But for GPUs to be considered as co-processor it consumes way too much power out of the total power budget. ough GPUs have such high o-chip memory bandwidth, accessing the o-chip global memory is very expensive(latency of around ¦þþ to þþ clock cycles). GPUs hide this latency by scheduling very large number of threads at a time. For memory bound application it becomes hard to hide this latency and they have slack. On CPUs we can take advantage of this by applying DVFS to reduce the dynamic power. Õ ó hZ£±u§ Õ. ±§o¶h± In this thesis work we investigate the power and performance characteristics of a GPU with DVFS(Dynamic voltage and frequency scaling). To help us achieve that we make use of microbenchmarks with dierent levels of memory boundedness and threads. We follow it up by applying DVFS to some more benchmarks. Apart from the introduction in this chapter the thesis is structured as follows. In Chapter ó , we discuss about GPU programming, GPU architecture, the concept behind DVFS based power savings and also some previous work. In Chapter ì, we describe the method used to measure power and implement DVFS on GPU. We also throw some light on the benchmarks that were used for the evaluation. In Chapter ¦, we evaluate the performance and power consumption of the benchmarks under dierent number of threads, shader frequencies and memory bandwidth. In Chapter ¢, we reect on the conclusions drawn from our evaluation. In Appendix: Assembly code, we provide the assembly code of the micro benchmarks we used. In Appendix: Power measurements, we provide all the execution times and power consumption readings from the evaluation. 2 hZ£±u§ BZh§¶o ó.Õ h¶oZ £§§Z ou CUDA is a GPGPU programming language that can be used to write programs for NVIDIA GPUs. It is an extension of the C programming language. A code written in CUDA consists of a mix of host(CPU) code and device(GPU) code[É]. e NVCC compiler compiles the device code and separates it from the host code. At a later stage a C compiler can be used to compile the host code and the device code parts are replaced by calls to the compiled device code. A data parallel function that is to be run on a device is called a kernel. A kernel creates many threads to compute this data parallel workload. A thread block consists of many threads and a group of thread block form a grid. Õ __global__ void SimpleKernel(float A[N],float B[N],float C[N], int N) ó { ì //calculate thread id ¦ int i= blockIDx.x * blockDim.x + threadIdx.x; ¢ if(i<N) ä C[i] = A[i] + B[i]; ß } É int main() Õþ { ÕÕ ...... Õó //kernel invocation Õì SimpleKernel<<<numberOfBlocks,threadsPerBlock>>>(A,B,C,N); Õ¦ ...... Õ¢ } Listing ó.Õ: A simple CUDA kernel ì ¦ hZ£±u§ ó. fZh§¶o In the Listing ó.Õ on page ì we have shown a very simple CUDA program model. Inside the kernel SimpleKernel we have CUDA specic variables that help us identify the data element for the particular thread. threadIdx identies a thread inside a block. Each thread inside a block has a unique thread id. blockIdx identies the block in a grid. e blockDim gives the size of a block in a particular dimension. When we invoke the kernel from the device code, along with the kernel arguments we also pass the thread details. e variable threadsPerBlock gives the number of threads in a block and numberOfBlocks indicate the number of blocks that are to be passed. All the blocks are of the same size(i.e. they have the same number of threads). Both numberOfBlocks and threadsPerBlock can be ÕD or óD or ìD data types. A more detailed explanation about CUDA programming can be found in [É]. ó.ó £¶ Z§h±uh±¶§u SM SM SM SM SM SM SM SM SM SM SM SM SM SM SM SM Fermi GPU Figure ó.Õ: On the le we have the CUDA core. SM is shown in the centre. It is made up of CUDA cores. e Fermi GPU architecture shown on the right has many SMs. Source: NVIDIA Fermi architecture whitepaper In our setup we use the Nvidia GTX¢þ GPU with CUDA compute capability ó.þ. It is based on the Fermi architecture[Õ¢]. Figure ó.Õ gives an overview of the Fermi architecture. e basic computing unit is the CUDA core. ere are ¢Õó CUDA cores on the Fermi. Each CUDA core can execute integer or oating point instructions for a single thread. ìó CUDA cores make up a single Streaming Multiprocessor(SM). ere in total Õä SMs on the GTX¢þ. Each SM has its own congurable shared memory and LÕ cache, which can be congured as Õä/¦ KB.
Load more

Recommended publications
  • CUDA by Example
    CUDA by Example AN INTRODUCTION TO GENERAL-PURPOSE GPU PROGRAMMING JASON SaNDERS EDWARD KANDROT Upper Saddle River, NJ • Boston • Indianapolis • San Francisco New York • Toronto • Montreal • London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City Sanders_book.indb 3 6/12/10 3:15:14 PM Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. NVIDIA makes no warranty or representation that the techniques described herein are free from any Intellectual Property claims. The reader assumes all risk of any such claims based on his or her use of these techniques. The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact: U.S. Corporate and Government Sales (800) 382-3419 [email protected] For sales outside the United States, please contact: International Sales [email protected] Visit us on the Web: informit.com/aw Library of Congress Cataloging-in-Publication Data Sanders, Jason.
    [Show full text]
  • Bitfusion Guide to CUDA Installation Bitfusion Guides Bitfusion: Bitfusion Guide to CUDA Installation
    WHITE PAPER–OCTOBER 2019 Bitfusion Guide to CUDA Installation Bitfusion Guides Bitfusion: Bitfusion Guide to CUDA Installation Table of Contents Purpose 3 Introduction 3 Some Sources of Confusion 4 So Many Prerequisites 4 Installing the NVIDIA Repository 4 Installing the NVIDIA Driver 6 Installing NVIDIA CUDA 6 Installing cuDNN 7 Speed It Up! 8 Upgrading CUDA 9 WHITE PAPER | 2 Bitfusion: Bitfusion Guide to CUDA Installation Purpose Bitfusion FlexDirect provides a GPU virtualization solution. It allows you to use the GPUs, or even partial GPUs (e.g., half of a GPU, or a quarter, or one-third of a GPU), on different servers as if they were attached to your local machine, a machine on which you can now run a CUDA application even though it has no GPUs of its own. Since Bitfusion FlexDirect software and ML/AI (machine learning/artificial intelligence) applications require other CUDA libraries and drivers, this document describes how to install the prerequisite software for both Bitfusion FlexDirect and for various ML/AI applications. If you already have your CUDA drivers and libraries installed, you can skip ahead to the chapter on installing Bitfusion FlexDirect. This document gives instruction and examples for Ubuntu, Red Hat, and CentOS systems. Introduction Bitfusion’s software tool is called FlexDirect. FlexDirect easily installs with a single command, but installing the third-party prerequisite drivers and libraries, plus any application and its prerequisite software, can seem a Gordian Knot to users new to CUDA code and ML/AI applications. We will begin untangling the knot with a table of the components involved.
    [Show full text]
  • Parallel Architectures and Algorithms for Large-Scale Nonlinear Programming
    Parallel Architectures and Algorithms for Large-Scale Nonlinear Programming Carl D. Laird Associate Professor, School of Chemical Engineering, Purdue University Faculty Fellow, Mary Kay O’Connor Process Safety Center Laird Research Group: http://allthingsoptimal.com Landscape of Scientific Computing 10$ 1$ 50%/year 20%/year Clock&Rate&(GHz)& 0.1$ 0.01$ 1980$ 1985$ 1990$ 1995$ 2000$ 2005$ 2010$ Year& [Steven Edwards, Columbia University] 2 Landscape of Scientific Computing Clock-rate, the source of past speed improvements have stagnated. Hardware manufacturers are shifting their focus to energy efficiency (mobile, large10" data centers) and parallel architectures. 12" 10" 1" 8" 6" #"of"Cores" Clock"Rate"(GHz)" 0.1" 4" 2" 0.01" 0" 1980" 1985" 1990" 1995" 2000" 2005" 2010" Year" 3 “… over a 15-year span… [problem] calculations improved by a factor of 43 million. [A] factor of roughly 1,000 was attributable to faster processor speeds, … [yet] a factor of 43,000 was due to improvements in the efficiency of software algorithms.” - attributed to Martin Grotschel [Steve Lohr, “Software Progress Beats Moore’s Law”, New York Times, March 7, 2011] Landscape of Computing 5 Landscape of Computing High-Performance Parallel Architectures Multi-core, Grid, HPC clusters, Specialized Architectures (GPU) 6 Landscape of Computing High-Performance Parallel Architectures Multi-core, Grid, HPC clusters, Specialized Architectures (GPU) 7 Landscape of Computing data VM iOS and Android device sales High-Performance have surpassed PC Parallel Architectures iPhone performance
    [Show full text]
  • (GPU) Computing
    Graphics Processing Unit (GPU) computing This section describes the graphics processing unit (GPU) computing feature of OptiSystem. Note: The GPU computing feature is only configurable with OptiSystem Version 11 (or higher) What is GPU computing? GPU computing or GPGPU takes advantage of a computer’s grahics processing card to augment the speed of general purpose scientific and engineering computing tasks. Compute Unified Device Architecture (CUDA) implementation for OptiSystem NVIDIA revolutionized the GPGPU and accelerated computing when it introduced a new parallel computing architecture: Compute Unified Device Architecture (CUDA). CUDA is both a hardware and software architecture for issuing and managing computations within the GPU, thus allowing it to operate as a generic data-parallel computing device. CUDA allows the programmer to take advantage of the parallel computing power of an NVIDIA graphics card to perform general purpose computations. OptiSystem CUDA implementation The OptiSystem model for GPU computing involves using a central processing unit (CPU) and GPU together in a heterogeneous co-processing computing model. The sequential part of the application runs on the CPU and the computationally-intensive part is accelerated by the GPU. In the OptiSystem GPU programming model, the application has been modified to map the compute-intensive kernels to the GPU. The remainder of the application remains within the CPU. CUDA parallel computing architecture The NVIDIA CUDA parallel computing architecture is enabled on GeForce®, Quadro®, and Tesla™ products. Whereas GeForce and Quadro are designed for consumer graphics and professional visualization respectively, the Tesla product family is designed ground-up for parallel computing and offers exclusive computing 3 GRAPHICS PROCESSING UNIT (GPU) COMPUTING features, and is the recommended choice for the OptiSystem GPU.
    [Show full text]
  • Massively Parallel Computing with CUDA
    Massively Parallel Computing with CUDA Antonino Tumeo Politecnico di Milano 1 GPUs have evolved to the point where many real world applications are easily implemented on them and run significantly faster than on multi-core systems. Future computing architectures will be hybrid systems with parallel-core GPUs working in tandem with multi-core CPUs. Jack Dongarra Professor, University of Tennessee; Author of “Linpack” Why Use the GPU? • The GPU has evolved into a very flexible and powerful processor: • It’s programmable using high-level languages • It supports 32-bit and 64-bit floating point IEEE-754 precision • It offers lots of GFLOPS: • GPU in every PC and workstation What is behind such an Evolution? • The GPU is specialized for compute-intensive, highly parallel computation (exactly what graphics rendering is about) • So, more transistors can be devoted to data processing rather than data caching and flow control ALU ALU Control ALU ALU Cache DRAM DRAM CPU GPU • The fast-growing video game industry exerts strong economic pressure that forces constant innovation GPUs • Each NVIDIA GPU has 240 parallel cores NVIDIA GPU • Within each core 1.4 Billion Transistors • Floating point unit • Logic unit (add, sub, mul, madd) • Move, compare unit • Branch unit • Cores managed by thread manager • Thread manager can spawn and manage 12,000+ threads per core 1 Teraflop of processing power • Zero overhead thread switching Heterogeneous Computing Domains Graphics Massive Data GPU Parallelism (Parallel Computing) Instruction CPU Level (Sequential
    [Show full text]
  • CUDA Flux: a Lightweight Instruction Profiler for CUDA Applications
    CUDA Flux: A Lightweight Instruction Profiler for CUDA Applications Lorenz Braun Holger Froning¨ Institute of Computer Engineering Institute of Computer Engineering Heidelberg University Heidelberg University Heidelberg, Germany Heidelberg, Germany [email protected] [email protected] Abstract—GPUs are powerful, massively parallel processors, hardware, it lacks verbosity when it comes to analyzing the which require a vast amount of thread parallelism to keep their different types of instructions executed. This problem is aggra- thousands of execution units busy, and to tolerate latency when vated by the recently frequent introduction of new instructions, accessing its high-throughput memory system. Understanding the behavior of massively threaded GPU programs can be for instance floating point operations with reduced precision difficult, even though recent GPUs provide an abundance of or special function operations including Tensor Cores for ma- hardware performance counters, which collect statistics about chine learning. Similarly, information on the use of vectorized certain events. Profiling tools that assist the user in such analysis operations is often lost. Characterizing workloads on PTX for their GPUs, like NVIDIA’s nvprof and cupti, are state-of- level is desirable as PTX allows us to determine the exact types the-art. However, instrumentation based on reading hardware performance counters can be slow, in particular when the number of the instructions a kernel executes. On the contrary, nvprof of metrics is large. Furthermore, the results can be inaccurate as metrics are based on a lower-level representation called SASS. instructions are grouped to match the available set of hardware Still, even if there was an exact mapping, some information on counters.
    [Show full text]
  • An Introduction to Gpus, CUDA and Opencl
    An Introduction to GPUs, CUDA and OpenCL Bryan Catanzaro, NVIDIA Research Overview ¡ Heterogeneous parallel computing ¡ The CUDA and OpenCL programming models ¡ Writing efficient CUDA code ¡ Thrust: making CUDA C++ productive 2/54 Heterogeneous Parallel Computing Latency-Optimized Throughput- CPU Optimized GPU Fast Serial Scalable Parallel Processing Processing 3/54 Why do we need heterogeneity? ¡ Why not just use latency optimized processors? § Once you decide to go parallel, why not go all the way § And reap more benefits ¡ For many applications, throughput optimized processors are more efficient: faster and use less power § Advantages can be fairly significant 4/54 Why Heterogeneity? ¡ Different goals produce different designs § Throughput optimized: assume work load is highly parallel § Latency optimized: assume work load is mostly sequential ¡ To minimize latency eXperienced by 1 thread: § lots of big on-chip caches § sophisticated control ¡ To maXimize throughput of all threads: § multithreading can hide latency … so skip the big caches § simpler control, cost amortized over ALUs via SIMD 5/54 Latency vs. Throughput Specificaons Westmere-EP Fermi (Tesla C2050) 6 cores, 2 issue, 14 SMs, 2 issue, 16 Processing Elements 4 way SIMD way SIMD @3.46 GHz @1.15 GHz 6 cores, 2 threads, 4 14 SMs, 48 SIMD Resident Strands/ way SIMD: vectors, 32 way Westmere-EP (32nm) Threads (max) SIMD: 48 strands 21504 threads SP GFLOP/s 166 1030 Memory Bandwidth 32 GB/s 144 GB/s Register File ~6 kB 1.75 MB Local Store/L1 Cache 192 kB 896 kB L2 Cache 1.5 MB 0.75 MB
    [Show full text]
  • Cuda C Programming Guide
    CUDA C PROGRAMMING GUIDE PG-02829-001_v10.0 | October 2018 Design Guide CHANGES FROM VERSION 9.0 ‣ Documented restriction that operator-overloads cannot be __global__ functions in Operator Function. ‣ Removed guidance to break 8-byte shuffles into two 4-byte instructions. 8-byte shuffle variants are provided since CUDA 9.0. See Warp Shuffle Functions. ‣ Passing __restrict__ references to __global__ functions is now supported. Updated comment in __global__ functions and function templates. ‣ Documented CUDA_ENABLE_CRC_CHECK in CUDA Environment Variables. ‣ Warp matrix functions now support matrix products with m=32, n=8, k=16 and m=8, n=32, k=16 in addition to m=n=k=16. www.nvidia.com CUDA C Programming Guide PG-02829-001_v10.0 | ii TABLE OF CONTENTS Chapter 1. Introduction.........................................................................................1 1.1. From Graphics Processing to General Purpose Parallel Computing............................... 1 1.2. CUDA®: A General-Purpose Parallel Computing Platform and Programming Model.............3 1.3. A Scalable Programming Model.........................................................................4 1.4. Document Structure...................................................................................... 5 Chapter 2. Programming Model............................................................................... 7 2.1. Kernels......................................................................................................7 2.2. Thread Hierarchy........................................................................................
    [Show full text]
  • CUDA 11 and A100 - WHAT’S NEW? Markus Hrywniak, 23Rd June 2020 TOPICS for TODAY
    CUDA 11 AND A100 - WHAT’S NEW? Markus Hrywniak, 23rd June 2020 TOPICS FOR TODAY Ampere architecture – A100, powering DGX–A100, HGX-A100... and soon, FZ Jülich‘s JUWELS Booster New CUDA 11 Toolkit release Overview of features Talk next week: Third generation Tensor Cores GTC talks go into much more details. See references! 2 HGX-A100 4-GPU HGX-A100 8-GPU • 4 A100 with NVLINK • 8 A100 with NVSwitch 3 HIERARCHY OF SCALES Multi-System Rack Multi-GPU System Multi-SM GPU Multi-Core SM Unlimited Scale 8 GPUs 108 Multiprocessors 2048 threads 4 AMDAHL’S LAW serial section parallel section serial section Amdahl’s Law parallel section Shortest possible runtime is sum of serial section times Time saved serial section Some Parallelism Increased Parallelism Infinite Parallelism Program time = Parallel sections take less time Parallel sections take no time sum(serial times + parallel times) Serial sections take same time Serial sections take same time 5 OVERCOMING AMDAHL: ASYNCHRONY & LATENCY serial section parallel section serial section Split up serial & parallel components parallel section serial section Some Parallelism Task Parallelism Infinite Parallelism Program time = Parallel sections overlap with serial sections Parallel sections take no time sum(serial times + parallel times) Serial sections take same time 6 OVERCOMING AMDAHL: ASYNCHRONY & LATENCY CUDA Concurrency Mechanisms At Every Scope CUDA Kernel Threads, Warps, Blocks, Barriers Application CUDA Streams, CUDA Graphs Node Multi-Process Service, GPU-Direct System NCCL, CUDA-Aware MPI, NVSHMEM 7 OVERCOMING AMDAHL: ASYNCHRONY & LATENCY Execution Overheads Non-productive latencies (waste) Operation Latency Network latencies Memory read/write File I/O ..
    [Show full text]
  • NVIDIA Ampere GA102 GPU Architecture Whitepaper
    NVIDIA AMPERE GA102 GPU ARCHITECTURE Second-Generation RTX Updated with NVIDIA RTX A6000 and NVIDIA A40 Information V2.0 Table of Contents Introduction 5 GA102 Key Features 7 2x FP32 Processing 7 Second-Generation RT Core 7 Third-Generation Tensor Cores 8 GDDR6X and GDDR6 Memory 8 Third-Generation NVLink® 8 PCIe Gen 4 9 Ampere GPU Architecture In-Depth 10 GPC, TPC, and SM High-Level Architecture 10 ROP Optimizations 11 GA10x SM Architecture 11 2x FP32 Throughput 12 Larger and Faster Unified Shared Memory and L1 Data Cache 13 Performance Per Watt 16 Second-Generation Ray Tracing Engine in GA10x GPUs 17 Ampere Architecture RTX Processors in Action 19 GA10x GPU Hardware Acceleration for Ray-Traced Motion Blur 20 Third-Generation Tensor Cores in GA10x GPUs 24 Comparison of Turing vs GA10x GPU Tensor Cores 24 NVIDIA Ampere Architecture Tensor Cores Support New DL Data Types 26 Fine-Grained Structured Sparsity 26 NVIDIA DLSS 8K 28 GDDR6X Memory 30 RTX IO 32 Introducing NVIDIA RTX IO 33 How NVIDIA RTX IO Works 34 Display and Video Engine 38 DisplayPort 1.4a with DSC 1.2a 38 HDMI 2.1 with DSC 1.2a 38 Fifth Generation NVDEC - Hardware-Accelerated Video Decoding 39 AV1 Hardware Decode 40 Seventh Generation NVENC - Hardware-Accelerated Video Encoding 40 NVIDIA Ampere GA102 GPU Architecture ii Conclusion 42 Appendix A - Additional GeForce GA10x GPU Specifications 44 GeForce RTX 3090 44 GeForce RTX 3070 46 Appendix B - New Memory Error Detection and Replay (EDR) Technology 49 Appendix C - RTX A6000 GPU Perf ormance 50 List of Figures Figure 1.
    [Show full text]
  • Gpgpu Processing in Cuda Architecture
    Advanced Computing: An International Journal ( ACIJ ), Vol.3, No.1, January 2012 GPGPU PROCESSING IN CUDA ARCHITECTURE Jayshree Ghorpade 1, Jitendra Parande 2, Madhura Kulkarni 3, Amit Bawaskar 4 1Departmentof Computer Engineering, MITCOE, Pune University, India [email protected] 2 SunGard Global Technologies, India [email protected] 3 Department of Computer Engineering, MITCOE, Pune University, India [email protected] 4Departmentof Computer Engineering, MITCOE, Pune University, India [email protected] ABSTRACT The future of computation is the Graphical Processing Unit, i.e. the GPU. The promise that the graphics cards have shown in the field of image processing and accelerated rendering of 3D scenes, and the computational capability that these GPUs possess, they are developing into great parallel computing units. It is quite simple to program a graphics processor to perform general parallel tasks. But after understanding the various architectural aspects of the graphics processor, it can be used to perform other taxing tasks as well. In this paper, we will show how CUDA can fully utilize the tremendous power of these GPUs. CUDA is NVIDIA’s parallel computing architecture. It enables dramatic increases in computing performance, by harnessing the power of the GPU. This paper talks about CUDA and its architecture. It takes us through a comparison of CUDA C/C++ with other parallel programming languages like OpenCL and DirectCompute. The paper also lists out the common myths about CUDA and how the future seems to be promising for CUDA. KEYWORDS GPU, GPGPU, thread, block, grid, GFLOPS, CUDA, OpenCL, DirectCompute, data parallelism, ALU 1. INTRODUCTION GPU computation has provided a huge edge over the CPU with respect to computation speed.
    [Show full text]
  • NVIDIA Tesla P100 Pcie Data Sheet
    NVIDIA® TESLA® P100 GPU ACCELERATOR World’s most advanced data center accelerator for PCIe-based servers HPC data centers need to support the ever-growing demands of scientists and researchers while staying within a tight budget. The old approach of deploying lots of commodity compute nodes requires huge interconnect overhead that substantially increases costs SPECIFICATIONS without proportionally increasing performance. GPU Architecture NVIDIA Pascal NVIDIA CUDA® Cores 3584 NVIDIA Tesla P100 GPU accelerators are the most advanced ever Double-Precision 4.7 TeraFLOPS built, powered by the breakthrough NVIDIA Pascal™ architecture Performance Single-Precision 9.3 TeraFLOPS and designed to boost throughput and save money for HPC and Performance hyperscale data centers. The newest addition to this family, Tesla Half-Precision 18.7 TeraFLOPS Performance P100 for PCIe enables a single node to replace half a rack of GPU Memory 16GB CoWoS HBM2 at commodity CPU nodes by delivering lightning-fast performance in a 732 GB/s or 12GB CoWoS HBM2 at broad range of HPC applications. 549 GB/s System Interface PCIe Gen3 MASSIVE LEAP IN PERFORMANCE Max Power Consumption 250 W ECC Yes NVIDIA Tesla P100 for PCIe Performance Thermal Solution Passive Form Factor PCIe Full Height/Length 30 X 2X K80 2X P100 (PIe) 4X P100 (PIe) Compute APIs CUDA, DirectCompute, 25 X OpenCL™, OpenACC ™ 20 X TeraFLOPS measurements with NVIDIA GPU Boost technology 15 X 10 X 5 X Appl cat on Speed-up 0 X NAMD VASP MIL HOOMD- AMBER affe/ Blue AlexNet Dual PU server, Intel E5-2698 v3 23 Hz, 256 B System Memory, Pre-Product on Tesla P100 TESLA P100 PCle | DATA SHEEt | Oct16 A GIANT LEAP IN PERFORMANCE Tesla P100 for PCIe is reimagined from silicon to software, crafted with innovation at every level.
    [Show full text]