#WWDC18 •Metal for Ray Tracing Acceleration • Session 606 Sean James, GPU Software Engineer Wayne Lister, GPU Software Engineer © 2018 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple. Metal Performance Shaders GPU-accelerated primitives, optimized for iOS and macOS • Image processing • Linear algebra • Machine learning—inference Using Metal 2 for Compute WWDC 2017 What’s New in Metal, Part 2 WWDC 2016 Metal Performance Shaders NEW GPU-accelerated primitives, optimized for iOS and macOS • Image processing • Linear algebra • Machine learning—inference and training Metal for Accelerating Machine Learning Hall 3 Thursday 4:00PM Metal Performance Shaders NEW GPU-accelerated primitives, optimized for iOS and macOS • Image processing • Linear algebra • Machine learning—inference and training • Ray tracing Metal for Accelerating Machine Learning Hall 3 Thursday 4:00PM Ray Tracing Tracing a ray’s path as it interacts with a scene Ray Tracing Tracing a ray’s path as it interacts with a scene Ray Tracing Tracing a ray’s path as it interacts with a scene Applications • Rendering • Audio and physics simulation • Collision detection • AI and pathfinding Rasterization Projects triangles onto the screen one at a time Rasterization Projects triangles onto the screen one at a time Fast—method of choice for games and real-time applications Rasterization Projects triangles onto the screen one at a time Fast—method of choice for games and real-time applications Difficult to model behavior of light Ray Tracing Reflections Can be computed accurately with ray tracing Amazon Lumberyard Bistro © 2017 Amazon.com, https://developer.nvidia.com/orca/amazon-lumberyard-bistro Ray traced with Metal, https://creativecommons.org/licenses/by/4.0/ Ray Tracing Soft shadows Can be computed directly with ray tracing Realistic transition from hard to soft shadows Ray Tracing Global illumination Naturally modeled with ray tracing Sponza Atrium © 2009 Frank Meinl, Crytek, https://casual-effects.com/data/ Ray traced with Metal, https://creativecommons.org/licenses/by/3.0/ Ray Tracing Many other effects—ambient occlusion, refraction, area lights, depth of field, motion blur Ray Tracing Many other effects—ambient occlusion, refraction, area lights, depth of field, motion blur Method of choice for photorealistic, offline rendering Ray Tracing Many other effects—ambient occlusion, refraction, area lights, depth of field, motion blur Method of choice for photorealistic, offline rendering Significantly more computationally expensive—doing more work to simulate physical effects Rendering with Ray Tracing Work backwards from camera to light source Rendering with Ray Tracing Work backwards from camera to light source Cast primary rays from the camera into the scene Rendering with Ray Tracing Work backwards from camera to light source Cast primary rays from the camera into the scene Compute shading at intersection points Rendering with Ray Tracing Compute direct lighting contribution Rendering with Ray Tracing Compute direct lighting contribution Cast shadow rays from intersection point to light source Rendering with Ray Tracing Compute direct lighting contribution Cast shadow rays from intersection point to light source In shadow if shadow ray does not reach light Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Add direct lighting reflected from secondary intersection point Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Add direct lighting reflected from secondary intersection point Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Add direct lighting reflected from secondary intersection point Repeat to simulate light bouncing Rendering with Ray Tracing Ray count grows exponentially with number of bounces Rendering with Ray Tracing Ray count grows exponentially with number of bounces Avoid growth—one shadow ray and secondary ray per bounce Rendering with Ray Tracing Ray count grows exponentially with number of bounces Avoid growth—one shadow ray and secondary ray per bounce Rendering with Ray Tracing Ray count grows exponentially with number of bounces Avoid growth—one shadow ray and secondary ray per bounce Average over multiple frames Rendering with Ray Tracing Rendering with Ray Tracing Generate Primary Rays Rendering with Ray Tracing Primary Rays Generate Primary Rays Intersect with Scene Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays Image Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays Image MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector Accepts batches of rays in a Metal buffer Ray Buffer MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector Ray Buffer Intersection Accepts batches of rays in a Metal buffer Buffer Returns one intersection per ray MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector Ray Buffer Intersection Accepts batches of rays in a Metal buffer Buffer Returns one intersection per ray Command Buffer Encodes into a Metal command buffer Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection Accelerating Ray/Scene Intersection MPSRayIntersector Scene represented by triangle vertices in a vertex buffer Intersector Ray Buffer Intersection Buffer Vertex Buffer MPSRayIntersector Scene represented by triangle vertices in a vertex buffer Intersector Ray Buffer Intersection Build an acceleration structure over Buffer the vertex buffer Acceleration Structure Vertex Buffer MPSRayIntersector Scene represented by triangle vertices in a vertex buffer Intersector Ray Buffer Intersection Build an acceleration structure over Buffer the vertex buffer Acceleration Pass acceleration structure to intersector Structure Vertex Buffer Ray Tracing with Metal Performance Shaders Ray Tracing with Metal Performance Shaders Primary Rays and Shading Ray Tracing with Metal Performance Shaders Primary Rays and Shading Shadow Rays Ray Tracing with Metal Performance Shaders Primary Rays and Shading Shadow Rays Secondary Rays Ray Tracing with Metal Performance Shaders Primary Rays and Shading Shadow Rays Secondary Rays Primary Rays and Shading Primary Rays and Shading Intersector Primary Rays and Shading Intersector Acceleration Structure Vertex Buffer Primary Rays and Shading Generate Primary Rays Intersector Ray Buffer Acceleration Structure Vertex Buffer Primary Rays and Shading Generate Primary Rays Intersector Ray Buffer Intersection Buffer Acceleration Structure Vertex Buffer Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer Creating an Intersector Create an MPSRayIntersector with a Metal device: let intersector = MPSRayIntersector(device: device) Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer Creating an Acceleration Structure Create an MPSTriangleAccelerationStructure with a Metal device: let accelerationStructure = MPSTriangleAccelerationStructure(device: device) Creating an Acceleration Structure Create an MPSTriangleAccelerationStructure with a Metal device: let accelerationStructure = MPSTriangleAccelerationStructure(device: device) Assign vertex buffer: accelerationStructure.vertexBuffer = vertexBuffer accelerationStructure.triangleCount = triangleCount Creating an Acceleration Structure Create an MPSTriangleAccelerationStructure with a Metal device: let accelerationStructure = MPSTriangleAccelerationStructure(device: device) Assign vertex buffer: accelerationStructure.vertexBuffer = vertexBuffer accelerationStructure.triangleCount = triangleCount Build acceleration structure: accelerationStructure.rebuild() Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex