Real-time Algorithms Programmable Pipeline History Summary Realtime Computer Graphics on GPUs Introduction Jan Kolomazn´ık Department of Software and Computer Science Education Faculty of Mathematics and Physics Charles University in Prague March 3, 2021 1 / 55 Real-time Algorithms Programmable Pipeline History Summary Real-time Algorithms 2 / 55 Real-time Algorithms Programmable Pipeline History Summary REAL-TIME ALGORITHMS I Time Constrains: I Hard limit I Soft limit I CG examples: I Video frame rate I Cinema – 24 Hz I TV – 25 (50) Hz, 30 (60) Hz I Video games – 30–60 Hz I Virtual reality – frame rate doubled I Haptic rendering – 1 kHz 3 / 55 Real-time Algorithms Programmable Pipeline History Summary REAL-TIME ALGORITHMS I Time Constrains: I Hard limit I Soft limit I CG examples: I Video frame rate I Cinema – 24 Hz I TV – 25 (50) Hz, 30 (60) Hz I Video games – 30–60 Hz I Virtual reality – frame rate doubled I Haptic rendering – 1 kHz 4 / 55 Real-time Algorithms Programmable Pipeline History Summary REAL-TIME ALGORITHMS I Time Constrains: I Hard limit I Soft limit I CG examples: I Video frame rate I Cinema – 24 Hz I TV – 25 (50) Hz, 30 (60) Hz I Video games – 30–60 Hz I Virtual reality – frame rate doubled I Haptic rendering – 1 kHz 5 / 55 Real-time Algorithms Programmable Pipeline History Summary REAL-TIME ALGORITHMS I Time Constrains: I Hard limit I Soft limit I CG examples: I Video frame rate I Cinema – 24 Hz I TV – 25 (50) Hz, 30 (60) Hz I Video games – 30–60 Hz I Virtual reality – frame rate doubled I Haptic rendering – 1 kHz 6 / 55 Real-time Algorithms Programmable Pipeline History Summary REAL-TIME ALGORITHMS I Time Constrains: I Hard limit I Soft limit I CG examples: I Video frame rate I Cinema – 24 Hz I TV – 25 (50) Hz, 30 (60) Hz I Video games – 30–60 Hz I Virtual reality – frame rate doubled I Haptic rendering – 1 kHz 7 / 55 Real-time Algorithms Programmable Pipeline History Summary HOW TO ACHIEVE SPEED I Optimal algorithm (time complexity ?) I Approximations vs. precision requirements I Tuning for specific hardware I Specialized tools for hot spots – GPUs 8 / 55 Real-time Algorithms Programmable Pipeline History Summary HOW TO ACHIEVE SPEED I Optimal algorithm (time complexity ?) I Approximations vs. precision requirements I Tuning for specific hardware I Specialized tools for hot spots – GPUs 9 / 55 Real-time Algorithms Programmable Pipeline History Summary HOW TO ACHIEVE SPEED I Optimal algorithm (time complexity ?) I Approximations vs. precision requirements I Tuning for specific hardware I Specialized tools for hot spots – GPUs 10 / 55 Real-time Algorithms Programmable Pipeline History Summary HOW TO ACHIEVE SPEED I Optimal algorithm (time complexity ?) I Approximations vs. precision requirements I Tuning for specific hardware I Specialized tools for hot spots – GPUs 11 / 55 Real-time Algorithms Programmable Pipeline History Summary WHY GPU? 12 / 55 Real-time Algorithms Programmable Pipeline History Summary Programmable Pipeline 13 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 14 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 15 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 16 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 17 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 18 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 19 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from clip space, rasterize into screen space – generate fragments Fragment Shader I Custom fragment processing – lighting, texturing, . Per-Fragment Ops I Depth (Z-buffer) and stencil tests, color blending Framebuffer I Output image, Z-buffer, stencil buffer, double buffering, rendering to texture screen I Output on screen 20 / 55 Real-time Algorithms Programmable Pipeline History Summary PROGRAMMABLE PIPELINE Scene Description API I Boundary representation – vertices, edges, faces I Comunication (draw schedule, data transfers): App Primitive Processing VBOs $ GPU driver $ GPU Vertex Shader I Prepare inputs for the shader processing I Transform vertices into clip space Primitive Assembly ([x; y; z] 2 (−1; 1)3), bundle attributes (colors, normals) for further processing Rasterizer I Prepare primives for rasterization Early Depth Test I Process primitives from