NVIDIA RTX: Enabling Ray Tracing in Vulkan Nuno Subtil, Sr. Devtech Engineer / Eric Werness, Sr. System Software Engineer March 27, 2018 Ray tracing vs. rasterization How did we get here? Overview Real-time ray tracing applications in 2018 Exposing RTX through Vulkan 2 Ray Tracing vs. Rasterization ▪ Rasterization: evaluate one triangle at a time o Store and accumulate some of the output data, but discard most of it ▪ Computationally cheap, but “local” ▪ Amenable to hardware implementation 3 Ray Tracing vs. Rasterization ▪ Ray tracing: sampling the entire scene one ray at a time o Entire scene is evaluated o Can visit same primitive many times ▪ Extremely flexible ▪ Can be computationally intensive 4 Ray Tracing at NVIDIA Decade+ of R&D ▪ Real-time graphics: rasterization is king o GPUs evolve to enable powerful rasterization-based algorithms ▪ 2007: CUDA brings general purpose computing to GPUs o Ray tracing research focused on acceleration via GPGPU 5 OptiX: A General Purpose Ray Tracing Engine Parker et al, OptiX: A General Purpose Ray Tracing Engine, SIGGRAPH 2010 6 Today: OptiX and Pro GPU Rendering ▪ OptiX: General purpose GPU ray tracing SDK ▪ OptiX abstraction supports future innovation 7 NVIDIA RTX “NVIDIA RTX opens the door to make real-time ray tracing a reality!” — Kim Libreri, CTO, Epic Games “With NVIDIA GV100 GPUs and RTX, we can now do real-time ray tracing. It’s just fantastic!” — Sébastien Guichou, CTO of Isotropix8 NVIDIA RTX Applications NVIDIA DesignWorks NVIDIA GameWorks OptiX Raytracing Extensions DirectX Raytracing for CUDA for Vulkan for Microsoft DX12 NVIDIA® RTX™ Technology NVIDIA Volta GPU 9 Ray Traced Shadows Physically correct shadows Sharp and Contact Hardening 10 RTAO Ray Traced Ambient Occlusion SSAO Improved ambient occlusion detail with fewer artifacts 11 Ray Traced Reflections Physically-correct reflections on arbitrary surfaces 12 Ray Tracing: Not Just Graphics ▪ Physics, particle simulations ▪ Audio path tracing / simulation ▪ AI visibility queries 13 Exposing RTX through Vulkan VK_NV_raytracing 14 API design principles ▪ Proven ray tracing API concepts ▪ Hardware-agnostic ▪ Good fit with Vulkan API ▪ Implementable on compute functionality 15 Graphics Pipelines Rasterization Draw Vertex Fragment Scheduling Rasterization ROP Call Shading Shading Ray Tracing Ray Traversal & Scheduling Scheduling Shading Generation Intersection 16 Building the Pipeline Ray tracing building blocks ▪ Acceleration Structure o Abstraction for scene geometry, enables efficient scene traversal ▪ Ray tracing shader domains o Handle ray generation and termination, intersections, material shading ▪ Shader binding table o Links shaders and resources to be used during traversal ▪ Ray tracing Pipeline object 17 Acceleration Structure Concepts ▪ Ray tracing: testing one ray against all scene primitives ▪ Acceleration structures required for efficient operation 18 Acceleration Structure Concepts Top Level Acceleration Structure (AS) ▪ Dual-level acceleration structure Transform Transform Transform and shading and shading and shading ▪ Opaque (implementation-defined) information information information data structures Bottom Bottom ▪ Efficient build and update Level AS Level AS 19 Building Acceleration Structures Vertex & Index Build/Update Bottom Level AS Buffers Vertex & Index Build/Update Bottom Level AS Buffers Vertex & Index Build/Update Bottom Level AS Buffers 20 Building Acceleration Structures Bottom Level AS Xform / Shading Bottom Level AS Xform / Shading Build/Update Top Level AS Bottom Level AS Xform / Shading 21 Creating Acceleration struct VkAccelerationStructureCreateInfoNV { VkStructureType sType; Structures void *pNext; // top-level or bottom-level VkAccelerationStructureTypeNV type; VkAccelerationStructureCreateFlagsNV flags; // used when copying an existing object VkAccelerationStructureNV *pOriginal; ▪ New object type VkAccelerationStructureNV uint32_t numInstances; uint32_t numGeometries; const VkGeometryNV *pGeometries; }; ▪ New descriptor type for binding AS to shaders vkCreateAccelerationStructureNV( VkDevice device, const VkAccelerationStructureCreateInfoNV *info, const VkAllocationCallbacks *pAllocator, VkAccelerationStructureNV *pAccelerationStruct ); 22 Acceleration Structure Memory Management vkGetAccelerationStructureMemoryRequirementsNV( VkDevice device, const ▪ Backing store for AS is managed by application VkAccelerationStructureMemoryRequirementsInfoNV *pInfo, VkMemoryReqiurements2 *pMemoryRequirements ); ▪ Memory requirements will vary vkBindAccelerationStructureMemoryNV(…); 23 Acceleration Structure Build/Update vkCmdBuildAccelerationStructureNV( VkCommandBuffer cmdBuf, VkAccelerationStructureTypeNV type, uint32_t numInstances, const VkBuffer instanceData, const VkDeviceSize instanceOffset, ▪ Single command to build or update an AS uint32_t numGeometries, const VkGeometryNV *pGeometries, VkBuildAccelerationStructureFlagsNV flags, ▪ AS updates have restrictions for efficiency VkBool32 update, VkAccelerationStructureNV dst, VkAccelerationStructureNVsrc, VkBuffer scratch ); 24 Acceleration Structure Copy vkCmdCopyAccelerationStructure( ▪ Acceleration structures can be copied, optionally VkCommandBuffer cmdBuf, VkAccelerationStructure dst, compacting during the copy VkAccelerationStructure src, VkCopyAccelerationStructureFlags flags ); 25 Ray tracing shader domains Ray Generation traceNV() Intersection Any-Hit Any Hit Acceleration Structure Intersection Any-Hit Traversal Closest-Hit Intersection Ray Generation Hit? no yes Miss Closest Hit 26 Inter-shader communication ▪ Ray payload o Application-defined structure to pass data between hit stages and shader Ray Generation stage that spawned a ray traceNV() o Used to return final intersection Any Hit information to ray generation shader Acceleration Structure Traversal Intersection Hit? Any-hit: read/update payload no yes Closest Hit: Miss Closest Hit read/update Payload Ray Generation: Allocate and fill initial payload 27 Inter-shader communication ▪ Ray Attributes o Application-defined structure to pass intersection information from Ray Generation intersection shader to hit shaders traceNV() Any Hit Acceleration Structure Traversal Intersection Intersection: write attributes Hit? Any Hit: read attributes no yes Intersection: write attributes Miss Closest Hit Closest Hit: read attributes 28 Ray Generation Shaders Ray Generation ▪ Starting point for all ray tracing work traceNV() Any Hit Acceleration ▪ Simple 2D grid of threads launched Structure Traversal from host Intersection Hit? ▪ Traces rays, writes final output no yes Miss Closest Hit 29 Intersection Shaders Ray Generation traceNV() ▪ Compute ray intersections with app- Any Hit defined primitives Acceleration Structure Traversal Intersection ▪ Ray-triangle intersections built-in Hit? no yes Miss Closest Hit 30 Any-hit shader Ray Generation traceNV() ▪ Invoked after an intersection is found Any Hit Acceleration Structure Traversal ▪ Multiple intersections invoked in Intersection arbitrary order Hit? no yes Miss Closest Hit 31 Closest-hit shader Ray Generation ▪ Invoked on the closest intersection of traceNV() the ray Any Hit Acceleration Structure Traversal ▪ Can read attributes and trace rays to Intersection modify the payload Hit? no yes Miss Closest Hit 32 Miss shader Ray Generation ▪ Invoked if no hit is found and traceNV() accepted Any Hit Acceleration Structure Traversal Intersection ▪ Can trace rays and modify ray payload Hit? no yes Miss Closest Hit 33 Inter-shader Interface Mapping to GLSL ▪ Ray payload and intersection attributes: new rayPayloadNV { … } block decorations intersectionAttributesNV { … } layout (location= 1) rayPayloadNV firstInterface {…} layout (location= 2) ▪ traceNV() builtin may be called with different rayPayloadNV secondInterface {…} payload structures in a single shader traceNV(…, 1) // firstInterface traceNV(…, 2) // secondInterface 34 traceNV( accelerationStructure topLevel, uint rayFlags, GLSL Extensions uint cullMask, ▪ Built-in variable decorations: uint sbtRecordOffset, uint sbtRecordStride, o ID/index information uint missIndex, vec3 origin, o ray parameters, hit parameters float tmin, vec3 direction, o instance transforms float tmax, int payload ); ▪ New functions: reportIntersectionNV( o traceNV() traces a ray into the scene float hit, uint hitKind o reportIntersectionNV() outputs intersection ); information ignoreIntersectionNV(); o ignoreIntersectionNV() rejects an intersection terminateRayNV(); o terminateRayNV() terminates the current ray 35 Shader Binding Table ▪ A given ray query into the scene can hit any object o … with different kinds of material shaders o … requiring different kinds of resources (e.g., textures) o … and different parameters (e.g., transparency value) o … per object instance! 36 Shader Binding Table ▪ Array of opaque shader handles + application storage inside VkBuffer ▪ Indexing used to determine shaders / data used ▪ App-defined stride for SBT records to accommodate different resource sets 37 Shader Binding Table ShaderBindingTable Buffer Shader Custom Data & Shader Custom Data & ... ... Reference Indices Reference Indices Traversal yields instanceOffset RT Pipeline with RT Pipeline with Descriptor Sets Descriptor Sets RayGeneration ClosestHit Shader Shader reacts on Acceleration Structures Textures etc. traces rays hit (can trace) 38 vkCmdTraceRaysNV vkCmdTraceRaysNV( ▪ Entry point to invoke ray tracing work VkCommandBuffer cmdBuf, VkBuffer shaderBindingTable, uint32_t raygenShaderBindingTableIndex, uint32_t missShaderBindingTableIndex,