DOCSLIB.ORG

Cuda C Programming Guide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Cuda C Programming Guide CUDA C PROGRAMMING GUIDE PG-02829-001_v10.1 | August 2019 Design Guide CHANGES FROM VERSION 9.0 ‣ Documented restriction that operator-overloads cannot be __global__ functions in Operator Function. ‣ Removed guidance to break 8-byte shuffles into two 4-byte instructions. 8-byte shuffle variants are provided since CUDA 9.0. See Warp Shuffle Functions. ‣ Passing __restrict__ references to __global__ functions is now supported. Updated comment in __global__ functions and function templates. ‣ Documented CUDA_ENABLE_CRC_CHECK in CUDA Environment Variables. ‣ Warp matrix functions now support matrix products with m=32, n=8, k=16 and m=8, n=32, k=16 in addition to m=n=k=16. ‣ Added new Unified Memory sections: System Allocator, Hardware Coherency, Access Counters www.nvidia.com CUDA C Programming Guide PG-02829-001_v10.1 | ii TABLE OF CONTENTS Chapter 1. Introduction.........................................................................................1 1.1. From Graphics Processing to General Purpose Parallel Computing............................... 1 1.2. CUDA®: A General-Purpose Parallel Computing Platform and Programming Model.............3 1.3. A Scalable Programming Model.........................................................................4 1.4. Document Structure...................................................................................... 5 Chapter 2. Programming Model............................................................................... 7 2.1. Kernels......................................................................................................7 2.2. Thread Hierarchy......................................................................................... 8 2.3. Memory Hierarchy....................................................................................... 10 2.4. Heterogeneous Programming.......................................................................... 12 2.5. Compute Capability..................................................................................... 14 Chapter 3. Programming Interface..........................................................................15 3.1. Compilation with NVCC................................................................................ 15 3.1.1. Compilation Workflow.............................................................................16 3.1.1.1. Offline Compilation.......................................................................... 16 3.1.1.2. Just-in-Time Compilation....................................................................16 3.1.2. Binary Compatibility...............................................................................16 3.1.3. PTX Compatibility..................................................................................17 3.1.4. Application Compatibility.........................................................................17 3.1.5. C/C++ Compatibility............................................................................... 18 3.1.6. 64-Bit Compatibility............................................................................... 18 3.2. CUDA C Runtime.........................................................................................18 3.2.1. Initialization.........................................................................................19 3.2.2. Device Memory..................................................................................... 20 3.2.3. Shared Memory..................................................................................... 23 3.2.4. Page-Locked Host Memory........................................................................28 3.2.4.1. Portable Memory..............................................................................29 3.2.4.2. Write-Combining Memory....................................................................29 3.2.4.3. Mapped Memory...............................................................................30 3.2.5. Asynchronous Concurrent Execution............................................................ 31 3.2.5.1. Concurrent Execution between Host and Device........................................31 3.2.5.2. Concurrent Kernel Execution............................................................... 31 3.2.5.3. Overlap of Data Transfer and Kernel Execution......................................... 32 3.2.5.4. Concurrent Data Transfers.................................................................. 32 3.2.5.5. Streams.........................................................................................32 3.2.5.6. Graphs.......................................................................................... 36 3.2.5.7. Events...........................................................................................42 3.2.5.8. Synchronous Calls.............................................................................43 3.2.6. Multi-Device System............................................................................... 43 3.2.6.1. Device Enumeration.......................................................................... 43 www.nvidia.com CUDA C Programming Guide PG-02829-001_v10.1 | iii 3.2.6.2. Device Selection.............................................................................. 43 3.2.6.3. Stream and Event Behavior................................................................. 44 3.2.6.4. Peer-to-Peer Memory Access................................................................44 3.2.6.5. Peer-to-Peer Memory Copy..................................................................45 3.2.7. Unified Virtual Address Space................................................................... 46 3.2.8. Interprocess Communication..................................................................... 46 3.2.9. Error Checking......................................................................................47 3.2.10. Call Stack.......................................................................................... 48 3.2.11. Texture and Surface Memory................................................................... 48 3.2.11.1. Texture Memory............................................................................. 48 3.2.11.2. Surface Memory............................................................................. 58 3.2.11.3. CUDA Arrays..................................................................................62 3.2.11.4. Read/Write Coherency..................................................................... 62 3.2.12. Graphics Interoperability........................................................................62 3.2.12.1. OpenGL Interoperability................................................................... 63 3.2.12.2. Direct3D Interoperability...................................................................65 3.2.12.3. SLI Interoperability..........................................................................71 3.3. Versioning and Compatibility.......................................................................... 72 3.4. Compute Modes..........................................................................................73 3.5. Mode Switches........................................................................................... 74 3.6. Tesla Compute Cluster Mode for Windows.......................................................... 74 Chapter 4. Hardware Implementation......................................................................76 4.1. SIMT Architecture....................................................................................... 76 4.2. Hardware Multithreading...............................................................................78 Chapter 5. Performance Guidelines........................................................................ 80 5.1. Overall Performance Optimization Strategies...................................................... 80 5.2. Maximize Utilization.................................................................................... 80 5.2.1. Application Level...................................................................................80 5.2.2. Device Level........................................................................................ 81 5.2.3. Multiprocessor Level...............................................................................81 5.2.3.1. Occupancy Calculator........................................................................ 83 5.3. Maximize Memory Throughput........................................................................ 85 5.3.1. Data Transfer between Host and Device....................................................... 86 5.3.2. Device Memory Accesses..........................................................................87 5.4. Maximize Instruction Throughput.....................................................................91 5.4.1. Arithmetic Instructions............................................................................91 5.4.2. Control Flow Instructions......................................................................... 95 5.4.3. Synchronization Instruction.......................................................................96 Appendix A. CUDA-Enabled GPUs........................................................................... 97 Appendix B. C Language Extensions........................................................................98 B.1. Function Execution Space Specifiers.................................................................98
Load more

Recommended publications
  • Fortran 90 Overview
    1 Fortran 90 Overview J.E. Akin, Copyright 1998 This overview of Fortran 90 (F90) features is presented as a series of tables that illustrate the syntax and abilities of F90. Frequently comparisons are made to similar features in the C++ and F77 languages and to the Matlab environment. These tables show that F90 has significant improvements over F77 and matches or exceeds newer software capabilities found in C++ and Matlab for dynamic memory management, user defined data structures, matrix operations, operator definition and overloading, intrinsics for vector and parallel pro- cessors and the basic requirements for object-oriented programming. They are intended to serve as a condensed quick reference guide for programming in F90 and for understanding programs developed by others. List of Tables 1 Comment syntax . 4 2 Intrinsic data types of variables . 4 3 Arithmetic operators . 4 4 Relational operators (arithmetic and logical) . 5 5 Precedence pecking order . 5 6 Colon Operator Syntax and its Applications . 5 7 Mathematical functions . 6 8 Flow Control Statements . 7 9 Basic loop constructs . 7 10 IF Constructs . 8 11 Nested IF Constructs . 8 12 Logical IF-ELSE Constructs . 8 13 Logical IF-ELSE-IF Constructs . 8 14 Case Selection Constructs . 9 15 F90 Optional Logic Block Names . 9 16 GO TO Break-out of Nested Loops . 9 17 Skip a Single Loop Cycle . 10 18 Abort a Single Loop . 10 19 F90 DOs Named for Control . 10 20 Looping While a Condition is True . 11 21 Function definitions . 11 22 Arguments and return values of subprograms . 12 23 Defining and referring to global variables .
    [Show full text]
  • Quick Overview: Complex Numbers
    Quick Overview: Complex Numbers February 23, 2012 1 Initial Definitions Definition 1 The complex number z is defined as: z = a + bi (1) p where a, b are real numbers and i = −1. Remarks about the definition: • Engineers typically use j instead of i. • Examples of complex numbers: p 5 + 2i; 3 − 2i; 3; −5i • Powers of i: i2 = −1 i3 = −i i4 = 1 i5 = i i6 = −1 i7 = −i . • All real numbers are also complex (by taking b = 0). 2 Visualizing Complex Numbers A complex number is defined by it's two real numbers. If we have z = a + bi, then: Definition 2 The real part of a + bi is a, Re(z) = Re(a + bi) = a The imaginary part of a + bi is b, Im(z) = Im(a + bi) = b 1 Im(z) 4i 3i z = a + bi 2i r b 1i θ Re(z) a −1i Figure 1: Visualizing z = a + bi in the complex plane. Shown are the modulus (or length) r and the argument (or angle) θ. To visualize a complex number, we use the complex plane C, where the horizontal (or x-) axis is for the real part, and the vertical axis is for the imaginary part. That is, a + bi is plotted as the point (a; b). In Figure 1, we can see that it is also possible to represent the point a + bi, or (a; b) in polar form, by computing its modulus (or size), and angle (or argument): p r = jzj = a2 + b2 θ = arg(z) We have to be a bit careful defining φ, since there are many ways to write φ (and we could add multiples of 2π as well).
    [Show full text]
  • Fortran Math Special Functions Library
    IMSL® Fortran Math Special Functions Library Version 2021.0 Copyright 1970-2021 Rogue Wave Software, Inc., a Perforce company. Visual Numerics, IMSL, and PV-WAVE are registered trademarks of Rogue Wave Software, Inc., a Perforce company. IMPORTANT NOTICE: Information contained in this documentation is subject to change without notice. Use of this docu- ment is subject to the terms and conditions of a Rogue Wave Software License Agreement, including, without limitation, the Limited Warranty and Limitation of Liability. ACKNOWLEDGMENTS Use of the Documentation and implementation of any of its processes or techniques are the sole responsibility of the client, and Perforce Soft- ware, Inc., assumes no responsibility and will not be liable for any errors, omissions, damage, or loss that might result from any use or misuse of the Documentation PERFORCE SOFTWARE, INC. MAKES NO REPRESENTATION ABOUT THE SUITABILITY OF THE DOCUMENTATION. THE DOCU- MENTATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. PERFORCE SOFTWARE, INC. HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS WITH REGARD TO THE DOCUMENTATION, WHETHER EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PAR- TICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL PERFORCE SOFTWARE, INC. BE LIABLE, WHETHER IN CONTRACT, TORT, OR OTHERWISE, FOR ANY SPECIAL, CONSEQUENTIAL, INDIRECT, PUNITIVE, OR EXEMPLARY DAMAGES IN CONNECTION WITH THE USE OF THE DOCUMENTATION. The Documentation is subject to change at any time without notice. IMSL https://www.imsl.com/ Contents Introduction The IMSL Fortran Numerical Libraries . 1 Getting Started . 2 Finding the Right Routine . 3 Organization of the Documentation . 4 Naming Conventions .
    [Show full text]
  • IEEE Standard 754 for Binary Floating-Point Arithmetic
    Work in Progress: Lecture Notes on the Status of IEEE 754 October 1, 1997 3:36 am Lecture Notes on the Status of IEEE Standard 754 for Binary Floating-Point Arithmetic Prof. W. Kahan Elect. Eng. & Computer Science University of California Berkeley CA 94720-1776 Introduction: Twenty years ago anarchy threatened floating-point arithmetic. Over a dozen commercially significant arithmetics boasted diverse wordsizes, precisions, rounding procedures and over/underflow behaviors, and more were in the works. “Portable” software intended to reconcile that numerical diversity had become unbearably costly to develop. Thirteen years ago, when IEEE 754 became official, major microprocessor manufacturers had already adopted it despite the challenge it posed to implementors. With unprecedented altruism, hardware designers had risen to its challenge in the belief that they would ease and encourage a vast burgeoning of numerical software. They did succeed to a considerable extent. Anyway, rounding anomalies that preoccupied all of us in the 1970s afflict only CRAY X-MPs — J90s now. Now atrophy threatens features of IEEE 754 caught in a vicious circle: Those features lack support in programming languages and compilers, so those features are mishandled and/or practically unusable, so those features are little known and less in demand, and so those features lack support in programming languages and compilers. To help break that circle, those features are discussed in these notes under the following headings: Representable Numbers, Normal and Subnormal, Infinite
    [Show full text]
  • A Practical Introduction to Python Programming
    A Practical Introduction to Python Programming Brian Heinold Department of Mathematics and Computer Science Mount St. Mary’s University ii ©2012 Brian Heinold Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported Li- cense Contents I Basics1 1 Getting Started 3 1.1 Installing Python..............................................3 1.2 IDLE......................................................3 1.3 A first program...............................................4 1.4 Typing things in...............................................5 1.5 Getting input.................................................6 1.6 Printing....................................................6 1.7 Variables...................................................7 1.8 Exercises...................................................9 2 For loops 11 2.1 Examples................................................... 11 2.2 The loop variable.............................................. 13 2.3 The range function............................................ 13 2.4 A Trickier Example............................................. 14 2.5 Exercises................................................... 15 3 Numbers 19 3.1 Integers and Decimal Numbers.................................... 19 3.2 Math Operators............................................... 19 3.3 Order of operations............................................ 21 3.4 Random numbers............................................. 21 3.5 Math functions............................................... 21 3.6 Getting
    [Show full text]
  • A Fast-Start Method for Computing the Inverse Tangent
    A Fast-Start Method for Computing the Inverse Tangent Peter Markstein Hewlett-Packard Laboratories 1501 Page Mill Road Palo Alto, CA 94062, U.S.A. [email protected] Abstract duction method is handled out-of-loop because its use is rarely required.) In a search for an algorithm to compute atan(x) For short latency, it is often desirable to use many in- which has both low latency and few floating point in- structions. Different sequences may be optimal over var- structions, an interesting variant of familiar trigonom- ious subdomains of the function, and in each subdomain, etry formulas was discovered that allow the start of ar- instruction level parallelism, such as Estrin’s method of gument reduction to commence before any references to polynomial evaluation [3] reduces latency at the cost of tables stored in memory are needed. Low latency makes additional intermediate calculations. the method suitable for a closed subroutine, and few In searching for an inverse tangent algorithm, an un- floating point operations make the method advantageous usual method was discovered that permits a floating for a software-pipelined implementation. point argument reduction computation to begin at the first cycle, without first examining a table of values. While a table is used, the table access is overlapped with 1Introduction most of the argument reduction, helping to contain la- tency. The table is designed to allow a low degree odd Short latency and high throughput are two conflict- polynomial to complete the approximation. A new ap- ing goals in transcendental function evaluation algo- proach to the logarithm routine [1] which motivated this rithm design.
    [Show full text]
  • Appendix a Mathematical Fundamentals
    Appendix A Mathematical Fundamentals In this book, it is assumed that the reader has familiarity with the mathematical definitions from the areas of Linear Algebra, Trigonometry, and other related areas. Some of them that are introduced in this Appendix to make the reader quickly understand the derivations and notations used in different chapters of this book. A.1 Function ‘Atan2’ The usual inverse tangent function denoted by Atan(z), where z=y/x, returns an angle in the range (-π/2, π/2). In order to express the full range of angles it is useful to define the so called two- argument arctangent function denoted by Atan2(y,x), which returns angle in the entire range, i.e., (- π, π). This function is defined for all (x,y)≠0, and equals the unique angle θ such that x y cosθ = , and sinθ = …(A.1) x 2 + y 2 x 2 + y 2 The function uses the signs of x and y to select the appropriate quadrant for the angle θ, as explained in Table A.1. Table A.1 Evaluation of ‘Atan2’ function x Atan2(y,x) +ve Atan(z) 0 Sgn(y) π/2 -ve Atan(z)+ Sgn(y) π In Table A.1, z=y/x, and Sgn(.) denotes the usual sign function, i.e., its value is -1, 0, or 1 depending on the positive, zero, and negative values of y, respectively. However, if both x and y are zeros, ‘Atan2’ is undefined. Now, using the table, Atan2(-1,1)= -π/4 and Atan2(1,-1)= 3π/4, whereas the Atan(-1) returns -π/4 in both the cases.
    [Show full text]
  • Hardware Implementations of Fixed-Point Atan2
    Hardware Implementations of Fixed-Point Atan2 Hardware Implementations of Fixed-Point Atan2 Florent de Dinechin Matei I¸stoan Universit´ede Lyon, INRIA, INSA-Lyon, CITI-Lab ARITH22 Florent de Dinechin, Matei I¸stoan Hardware Implementations of Fixed-Point Atan2 Hardware Implementations of Fixed-Point Atan2 Introduction: Methods for computing Atan2 Methods for Computing atan2 in Hardware Yet another arithmetic function . • . that is useful in telecom (to recover the phase of a signal) (12{24 bits of precision) • . and in general for cartesian to polar coordinate transformation • and an interesting function, nonetheless 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1 0.8 1 0.6 0.8 0.4 x 0.6 0.4 0.2 0.2 y 0 0 Florent de Dinechin, Matei I¸stoan Hardware Implementations of Fixed-Point Atan2 ARITH22 2 / 24 Hardware Implementations of Fixed-Point Atan2 Introduction: Methods for computing Atan2 Common Specification • target function y 1 (x; y) 1 y α = atan2(y; x) f (x; y) = arctan ( ) π x −1 1 x • input: fixed-point format −1 −1 0 1 ky y arctan ( ) = arctan ( ) [ ) kx x • output: fixed-point format and binary angles y (0; 1) π 2 −1 0 1 (−1; 0) π 0 (1; 0) −π x =) [ ) Florent de Dinechin, Matei I¸stoan Hardware Implementations of Fixed-Point Atan2 ARITH22 3 / 24 Hardware Implementations of Fixed-Point Atan2 Introduction: Methods for computing Atan2 Common Specification • target function y 1 (x; y) 1 y α = atan2(y; x) f (x; y) = arctan ( ) π x −1 1 x • input: fixed-point format −1 −1 0 1 ky y arctan ( ) = arctan ( ) [ ) kx x • output: fixed-point format and binary
    [Show full text]
  • Taichi Documentation Release 0.6.26
    taichi Documentation Release 0.6.26 Taichi Developers Aug 12, 2020 Overview 1 Why new programming language1 1.1 Design decisions.............................................1 2 Installation 3 2.1 Troubleshooting.............................................3 2.1.1 Windows issues.........................................3 2.1.2 Python issues..........................................3 2.1.3 CUDA issues..........................................4 2.1.4 OpenGL issues.........................................4 2.1.5 Linux issues...........................................5 2.1.6 Other issues...........................................5 3 Hello, world! 7 3.1 import taichi as ti.............................................8 3.2 Portability................................................8 3.3 Fields...................................................9 3.4 Functions and kernels..........................................9 3.5 Parallel for-loops............................................. 10 3.6 Interacting with other Python packages................................. 11 3.6.1 Python-scope data access.................................... 11 3.6.2 Sharing data with other packages................................ 11 4 Syntax 13 4.1 Taichi-scope vs Python-scope...................................... 13 4.2 Kernels.................................................. 13 4.2.1 Arguments........................................... 14 4.2.2 Return value........................................... 14 4.2.3 Advanced arguments...................................... 15 4.3 Functions................................................
    [Show full text]
  • Trigonometric Functions
    Title stata.com Trigonometric functions Contents Functions Reference Also see Contents acos(x) the radian value of the arccosine of x acosh(x) the inverse hyperbolic cosine of x asin(x) the radian value of the arcsine of x asinh(x) the inverse hyperbolic sine of x atan(x) the radian value of the arctangent of x atan2(y, x) the radian value of the arctangent of y=x, where the signs of the parameters y and x are used to determine the quadrant of the answer atanh(x) the inverse hyperbolic tangent of x cos(x) the cosine of x, where x is in radians cosh(x) the hyperbolic cosine of x sin(x) the sine of x, where x is in radians sinh(x) the hyperbolic sine of x tan(x) the tangent of x, where x is in radians tanh(x) the hyperbolic tangent of x Functions acos(x) Description: the radian value of the arccosine of x Domain: −1 to 1 Range: 0 to π acosh(x) Description: the inverse hyperbolic cosinep of x acosh(x) = ln(x + x2 − 1) Domain: 1 to 8.9e+307 Range: 0 to 709.77 asin(x) Description: the radian value of the arcsine of x Domain: −1 to 1 Range: −π=2 to π=2 asinh(x) Description: the inverse hyperbolic sinep of x asinh(x) = ln(x + x2 + 1) Domain: −8.9e+307 to 8.9e+307 Range: −709.77 to 709.77 1 2 Trigonometric functions atan(x) Description: the radian value of the arctangent of x Domain: −8e+307 to 8e+307 Range: −π=2 to π=2 atan2(y, x) Description: the radian value of the arctangent of y=x, where the signs of the parameters y and x are used to determine the quadrant of the answer Domain y: −8e+307 to 8e+307 Domain x: −8e+307 to 8e+307 Range: −π to
    [Show full text]
  • Hardware Implementations of Fixed-Point Atan2 Florent De Dinechin, Matei Istoan
    Hardware implementations of fixed-point Atan2 Florent de Dinechin, Matei Istoan To cite this version: Florent de Dinechin, Matei Istoan. Hardware implementations of fixed-point Atan2. 22nd IEEE Symposium on Computer Arithmetic, Jun 2015, Lyon, France. hal-01091138 HAL Id: hal-01091138 https://hal.inria.fr/hal-01091138 Submitted on 4 Dec 2014 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Distributed under a Creative Commons Attribution| 4.0 International License Hardware implementations of fixed-point Atan2 Florent de Dinechin, Matei Istoan Universite´ de Lyon, INRIA, INSA-Lyon, CITI-INRIA, F-69621, Villeurbanne, France Abstract—The atan2 function computes the polar angle y arctan(x/y) of a point given by its cartesian coordinates. 1 It is widely used in digital signal processing to recover (x,y) the phase of a signal. This article studies for this context α = atan2(y,x) the implementation of atan2 with fixed-point inputs and outputs. It compares the prevalent CORDIC shift-and-add -1 1 x algorithm to two multiplier-based techniques. The first one reduces the bivariate atan2 function to two functions of one variable: the reciprocal, and the arctangent.
    [Show full text]
  • [6.48] Use R▷P () for Four-Quadrant Arc Tangent Function
    [6.48] Use R▶P✕ () for four-quadrant arc tangent function The built-in arc tangent function, tan-1(), cannot return the correct quadrant of an angle specified by x- and y-coordinates, because the argument does not contain enough information. Suppose we want the angle between the x-axis and a ray with origin (0,0) and passing through point (1,1). Since y − tan( ) = we havetan( ) = 1 or= 1( ) so = ✜ ✕ x ✕ 1 ✕ tan 1 ✕ 4 However, if the ray instead passes through (-1,-1), we get the same result since (-1/-1) is also equal to 1, but the angle is actually (−3✜)/4. This difficulty was addressed by the designers of the Fortran programming language, which includes a function called atan2(y,x) to find the arc tangent of y/x correctly in any of the four quadrants, by accounting for the signs of x and y. It is a simple matter to accomplish this in TI Basic with the built-in R▶Pθ() function, if we account for the special case (0,0). In fact, it can be done with a single line of TI Basic code: when(x=0 and y=0,undef,R▶Pθ(x,y)) undef is returned for (0,0), otherwise R▶Pθ(0,0) returns itself in Radian angle mode, and this expression in Degree angle mode: 180*R▶Pθ(0,0)/Œ Neither of these results are useful. A more elaborate function can be written which also handles list and matrix arguments: atan2(αx,αy) Func ©(x,y) 4-quadrant arctan(y/x) ©Must be installed in math\ ©6jan02/[email protected] local αt,εm,τx © Function name, error message, αx type "atan2 error"→εm © Initialize error message define αt(α,β)=func © Function finds atan2() of simple elements
    [Show full text]