Vulkan Case Study 2016 Khronos Seoul DevU SAMSUNG Electronics Soowan Park Graphics Engineer ([email protected]) Joonyong Park Senior Graphics Engineer ([email protected]) Before the start All case study information & contents are based on our development experiences with Galaxy S7 spanning two chipset variants, using the ARM Mali and Qualcomm Adreno GPU. Samsung Electronics Who we are • GPU, Graphics R&D, MCD, SAMSUNG Electronics. • [email protected] Samsung Electronics What we did ProtoStar, HIT, NFS, Vainglory MWC, GDC, SDC, E3, Gamescom, CEDEC Samsung Electronics History Samsung Electronics History Samsung Electronics Agenda 1. Swapchain 2. Uniform Buffer 3. GPU Driver 4. Rendering 5. GLES Fall-back 6. Development Tip Samsung Electronics For who? For Android Vulkan Developer. It’s very simple case, But important! Samsung Electronics 1. Swapchain Swapchain - Android Triple Buffering - Google Project Butter (Applied since Android 4.1 Jelly Bean release) • Android OpenGL ES runs with triple buffering by default • adb shell dumpsys SurfaceFlinger → Image Count of Swapchain • Android platform requires at least 3 buffers to have better performance for this reason. #0 #1 #2 #0 #1 User can’t control the number of BackBuffer in OpenGL ES With Java SurfaceView • Currently Android Vulkan only support native activity. But, there are way to using SurfaceView & Java activity by passing surface handle to native through JNI to get NativeWindow handle. q.v. : https://developer.android.com/ndk/reference/group___native_activity.html • Recommend to using GLSurfaceView like separated java side Renderthread for main render loop. Samsung Electronics Swapchain - Presentation Mode • VK_PRESENT_MODE_MAILBOX_KHR Swapchain Images Internal queue (impl dependant) #0 #1 #2 X* vkAcquireNextImage vkAcquireNextImage vkAcquireNextImage vkQueuePresent vkQueuePresent vkQueuePresent #0 #1 #2 X=#0 X=#1 X=#2 Latency VBLANK Display controller will read from #1 Samsung Electronics Swapchain - Presentation Mode • VK_PRESENT_MODE_FIFO_KHR Swapchain Images Internal queue #0 #1 #2 X* Y* Z* vkAcquireNextImage vkAcquireNextImage vkAcquireNextImage vkQueuePresent vkQueuePresent vkQueuePresent #0 #1 #2 X=#0 Y=#1 Z=#2 Latency VBLANK Swaps #0 stored in X with the backbuffer. Samsung Electronics Swapchain - Presentation Mode VK_PRESENT_MODE_FIFO_KHR 60 FPS line VK_PRESENT_MODE_MAILBOX_KHR 60 FPS line ※ DO NOT use MAILBOX mode in game. Unless latency is critical and you know what you’re doing. Samsung Electronics Swapchain - Presentation Mode Code Level (q.v. : https://www.khronos.org/registry/vulkan/specs/1.0-wsi_extensions/xhtml/vkspec.html, , 29.5. Surface Queries) uint32_t presentModeCount = 0; vkGetPhysicalDeviceSurfacePresentModesKHR(physicalDevice, surface, &presentModeCount, VK_NULL_HANDLE); std::vector<VkPresentModeKHR> pPresentModes(presentModeCount); vkGetPhysicalDeviceSurfacePresentModesKHR(physicalDevice, surface, &presentModeCount, pPresentModes.data()); VkPresentModeKHR presentMode = VK_PRESENT_MODE_FIFO_KHR; const uint32_t desiredArraySize = 2; VkPresentModeKHR desiredPresentMode[] = { VK_PRESENT_MODE_FIFO_KHR, VK_PRESENT_MODE_MAILBOX_KHR }; for (int d_n = 0; d_n < desiredArraySize; ++d_n) { for (int p_n = 0; p_n < presentModeCount; ++p_n) { if (pPresentModes[p_n] == desiredPresentMode[d_n]) { presentMode = desiredPresentMode[d_n]; d_n = desiredArraySize; break; } } } Samsung Electronics Swapchain - SwapBuffer Comparison (Android) WSI (Windows System Integration) RENDERFRAME N (Vulkan) WILL BLOCK HERE COMMAND FLUSHING & RENDERING Recorded Into Command Buffer #0 vkAcquireNextImageKHR #0 vkQueueSubmit #0 vkQueuePresentKHR #0 associated Graphics Queue APPLICATION COMPLETE! Can explicitly get GPU rendering completion signal by using fence from submit VkImage(Buffer) #0 Rendering Complete Semaphore VkImage #0 VkImage (Buffer) #1 WindowBuffer INTERNAL WindowBuffer WAIT VkImage (Buffer) #2 Dequeue Queue SURFACE FLINGER Associated Native Window DISPLAY ※ Application does the “blocking wait” to sync with GPU.(VK_PRESENT_MODE_FIFO_KHR) RENDERFRAME N (OpenGL ES) WILL BLOCK HERE glClear / glDrawXXX #0 Render Into BackBuffer (FrameBuffer 0) #0 eglSwapBuffer #0 APPLICATION COMMAND FLUSHING & glFlush() #0 RENDERING EGLSurface : GfxBuffer #0 No way to get GPU rendering WindowBuffer EGLSurface : GfxBuffer #0 EGLSurface : GfxBuffer #1 completion WindowBuffer EGLSurface : Gfxbuffer #2 Dequeue Queue SURFACE FLINGER Associated Native Window DISPLAY Samsung Electronics Swapchain - Synchronization failed case • Tearing Samsung Electronics Swapchain - Synchronization • Fence Logic Swapchain Swapchain VkImage #0 VkImage #1 VkImage #2 VkImage #0 VkImage #1 VkImage #2 VkFence #0 VkFence #1 VkFence #2 VkFence #0 VkFence #1 VkFence #2 VkCommandBufferPool(Single-Thread) VkCommandBuffer #0 VkCommandBuffer #1 VkCommandBuffer #2 VkCommandBuffer #0 VkCommandBuffer #1 VkCommandBuffer #2 vkWaitForFences(fence #0) vkWaitForFences(fence #1) vkWaitForFences(fence #2) vkResetFence(fence #0) vkResetFence(fence #1) vkResetFence(fence #2) vkResetCommandBuffer(buf #0) vkResetCommandBuffer(buf #1) vkResetCommandBuffer(buf #2) vkBeginCommandBuffer(buf #0) vkBeginCommandBuffer(buf #1) vkBeginCommandBuffer(buf #2) Render ~ Render ~ Render ~ vkQueueSubmit(fence #0) vkQueueSubmit(fence #1) vkQueueSubmit(fence #2) vkQueuePresentKHR vkQueuePresentKHR vkQueuePresentKHR Samsung Electronics Image Layout - Swapchain • Transitioning to the correct image layout for presenting and rendering. • Very begin of drawing, after the first acquire • getSwapchainImagesKHR : VK_IMAGE_LAYOUT_UNDEFINED • VK_IMAGE_LAYOUT_GENERAL • Clear presentable image • Draw Routine • Acquire • VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL • Render • VK_IMAGE_LAYOUT_PRESENT_SRC_KHR • Present // Create Swapchain vkGetSwapchainImagesKHR(device, swapchain, &swapchainImageCount, pSwapchainImages); // VK_IMAGE_LAYOUT_UNDEFINED // Frame loop swapchainIndex = acquire(); if (firstAcquire) { setImagesLayout(pSwapchainImages, swapchainImageCount, VK_IMAGE_LAYOUT_GENERAL); clearImages(pSwapchainImages, swapchainImageCount); } setImageLayout(pSwapchainImages[swapchainIndex], VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL); /* Rendering */ setImageLayout(pSwapchainImages[swapchainIndex], VK_IMAGE_LAYOUT_PRESENT_SRC_KHR); present(swapchainIndex); Samsung Electronics Image Layout - Texture Texturing in Vulkan VkDescriptorSet VkDescriptorImageInfo VkImageView VkImage VkDeviceMemory VkSampler VK_TILING_LINEAR • Create with VK_IMAGE_LAYOUT_PREINITIALIZED • Set ImageData using vkMapMemory, vkUnmapMemory • VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL VK_TILING_OPTIMAL • Create with VK_IMAGE_LAYOUT_UNDEFINED • VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL • Set ImageData using Staging Buffer • VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL You can check format property like this // Get Image Format Property VkFormatProperties formatProperty; vkGetPhysicalDeviceFormatProperties(physicalDevice, imageFormat, &formatProperty); if (formatProperty.optimalTilingFeatures & VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT) /**/; else if (formatProperty.linearTilingFeatures & VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT) /**/; Samsung Electronics Image Layout - Texture • Why we should use Staging Buffer? VK_TILING_LINEAR VK_TILING_OPTIMAL Texels are laid out in memory in row- Texels are laid out in an implementation- major order, possibly with some dependent arrangement, for more padding on each row VkImage(VkDeviceMemory) optimal memory access VkImage(VkDeviceMemory) So you can access it with this eq. Common // (x,y,z,layer) are in texel coordinates address(x,y,z,layer) = layer*arrayPitch + z*depthPitch + y*rowPitch + x*texelSize + ? offset; Compressed // (x,y,z,layer) are in compressed texel block coordinates address(x,y,z,layer) = layer*arrayPitch + z*depthPitch + y*rowPitch + x*compressedTexelBlockByteSize + offset; Samsung Electronics Image Layout - Texture • How can use Staging Buffer? Image data Fill image data into the VkBuffer VkImage with VK_TILING_OPTIMAL VkCommandBuffer vkCmdCopyBufferToImage ? VkBuffer VkBuffer& stagingBuffer = getStagingBuffer(imageBufferSize); VkBufferImageCopy region = getRegionFromImage(image); DO NOT use VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT fillBuffer(stagingBuffer, pImageData); with VK_TILING_OPTIMAL. vkCmdCopyBufferToImage(commandBuffer, stagingBuffer, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &region); Samsung Electronics Image Layout - Framebuffer (OnlyForColor) • Bind for Attachment (transitioning Off-screen render target to input texture e.g. environment map.. Post-processing.. etc) • VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL • Bind for Texture • VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL // Initialize FrameBuffer createFrameBuffer(frameBuffer); // VK_IMAGE_LAYOUT_UNDEFINED setImageLayout(frameBuffer, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL); ……… //Bind FrameBuffer bindFrameBuffer(frameBuffer); setImageLayout(frameBuffer, VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL); /* Render into frameBuffer */ unbindFrameBuffer(frameBuffer); // And set Default Framebuffer setImageLayout(frameBuffer, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL); setTexture(frameBuffer, 0); /* Render into backbuffer */ Samsung Electronics Image Layout - Framebuffer (OnlyForColor) Framebuffer in Vulkan VkFramebuffer VkImageView VkImage VkDeviceMemory Off-screen #0 Original Scene VkImage #0 VkImage #0 VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL VkImage #1 VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL Rendering Off-screen #1 NormalMap for PostProcessing VkImage