NVIDIA TURING GPU ARCHITECTURE Graphics Reinvented WP-09183-001_v01 TABLE OF CONTENTS Introduction to the NVIDIA Turing Architecture ....................................................................1 NVIDIA Turing Key Features.......................................................................................................... 3 New Streaming Multiprocessor (SM) ....................................................................................... 3 Turing Tensor Cores ................................................................................................................. 4 Real-Time Ray Tracing Acceleration ......................................................................................... 4 New Shading Advancements .................................................................................................... 4 Mesh Shading ...................................................................................................................... 4 Variable Rate Shading (VRS) ................................................................................................ 5 Texture-Space Shading ........................................................................................................ 5 Multi-View Rendering (MVR)............................................................................................... 5 Deep Learning Features for Graphics ....................................................................................... 5 Deep Learning Features for Inference ...................................................................................... 6 GDDR6 High-Performance Memory Subsystem ....................................................................... 6 Second-Generation NVIDIA NVLink .......................................................................................... 6 USB-C and VirtualLink ............................................................................................................... 6 Turing GPU Architecture In-Depth ........................................................................................7 Turing TU102 GPU ........................................................................................................................ 7 Turing Streaming Multiprocessor (SM) Architecture .................................................................. 11 Turing Tensor Cores ............................................................................................................... 15 Turing Optimized for Datacenter Applications ........................................................................... 16 Turing Memory Architecture and Display Features .................................................................... 20 GDDR6 Memory Subsystem ................................................................................................... 20 L2 Cache and ROPs ................................................................................................................. 21 Turing Memory Compression ................................................................................................. 22 Video and Display Engine ....................................................................................................... 22 USB-C and VirtualLink ................................................................................................................. 24 NVLink Improves SLI ................................................................................................................... 24 Turing Ray Tracing Technology............................................................................................26 Turing RT Cores .......................................................................................................................... 31 NVIDIA NGX Technology .....................................................................................................34 NGX Software Architecture ........................................................................................................ 34 Deep Learning Super-Sampling (DLSS) ....................................................................................... 35 InPainting ................................................................................................................................... 38 AI Slow-Mo ............................................................................................................................. 39 AI Super Rez ........................................................................................................................... 39 NVIDIA Turing GPU Architecture WP-09183-001_v01 | ii Turing Advanced Shading Technologies ..............................................................................40 Mesh Shading ............................................................................................................................. 40 Variable Rate Shading ................................................................................................................. 43 Content Adaptive Shading ...................................................................................................... 45 Motion Adaptive Shading ....................................................................................................... 46 Foveated Rendering ............................................................................................................... 47 Texture Space Shading ............................................................................................................... 48 The Mechanics of TSS ............................................................................................................. 49 Multi-View Rendering ................................................................................................................. 51 Multi-View Rendering Use Cases ............................................................................................ 52 Resource Management and Binding Model ............................................................................... 54 Turing Features Enhance Virtual Reality ..............................................................................55 Conclusion ..........................................................................................................................57 Appendix A Turing TU104 GPU ............................................................................................58 Appendix B Turing TU106 GPU ...........................................................................................63 Appendix C RTX-OPS Description ........................................................................................66 The Hybrid Rendering Model ..................................................................................................... 66 RTX-OPS Workload-based Metric Explained ............................................................................... 67 Appendix D Ray Tracing Overview .......................................................................................69 Basic Ray Tracing Mechanics ...................................................................................................... 70 Bounding Volume Hierarchy .................................................................................................. 71 Denoising Filtering ...................................................................................................................... 73 Ray-Traced Shadows, Ambient Occlusion, and Reflections ........................................................ 73 NVIDIA Turing GPU Architecture WP-09183-001_v01 | iii LIST OF FIGURES Figure 1. Turing Reinvents Graphics ............................................................................................ 2 Figure 2. Turing TU102 Full GPU with 72 SM Units ..................................................................... 8 Figure 3. NVIDIA Turing TU102 GPU .......................................................................................... 10 Figure 4. Turing TU102/TU104/TU106 Streaming Multiprocessor (SM).................................... 12 Figure 5. Concurrent Execution of Floating Point and Integer Instructions in the Turing SM.... 13 Figure 6. New Shared Memory Architecture ............................................................................. 14 Figure 7. Turing Shading Performance Speedup versus Pascal on Many Different Workloads . 14 Figure 8. New Turing Tensor Cores Provide Multi-Precision for AI Inference............................ 16 Figure 9. Tesla T4 delivers up to 40X Higher Inference Performance ........................................ 17 Figure 10. Tesla T4 Delivers More than 50X the Energy Efficiency of CPU-based Inferencing .... 18 Figure 11. Turing GDDR6 ............................................................................................................. 21 Figure 12. 50% Higher Effective Bandwidth ................................................................................ 22 Figure 13. Video Feature Enhancements ..................................................................................... 23 Figure 14. NVLink Enables New SLI Display Topologies ............................................................... 25 Figure 15. SOL MAN from NVIDIA SOL Ray Tracing Demo (See Demo) ....................................... 27 Figure 16. Hybrid Rendering Pipeline .......................................................................................... 28 Figure 17. Details