Tools #WWDC14 Advanced Graphics and Animations for iOS Apps Session 419 ! Axel Wefers Michael Ingrassia iOS Software Engineer iOS Software Engineer © 2014 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple. What You Will Learn Core Animation pipeline Rendering concepts UIBlurEffect UIVibrancyEffect Profiling tools Case studies Technology Framework UIKit Core Animation OpenGL ES Core Graphics Graphics Hardware Core Animation Pipeline Axel Wefers iOS Software Engineer Core Animation Pipeline Application Core Animation Pipeline Application Core Animation Core Animation Pipeline Application Core Animation Render Server Core Animation Pipeline Application Core Animation Render Server Core Animation Core Animation Pipeline Application Core Animation Render Server Core Animation GPU Core Animation Pipeline Application Core Animation Render Server Core Animation GPU Display Core Animation Pipeline Application Core Animation Render Server Core Animation GPU Display 16.67 ms Core Animation Pipeline Application Core Animation Handle Events Render Server Core Animation GPU Display 16.67 ms Core Animation Pipeline Application Core Animation Commit Transaction Render Server Core Animation GPU Display 16.67 ms Core Animation Pipeline Application Core Animation Commit Transaction Render Server Decode Core Animation GPU Display 16.67 ms Core Animation Pipeline Application Core Animation Commit Transaction Render Server Decode Core Animation Draw Calls GPU Display 16.67 ms Core Animation Pipeline Application Core Animation Commit Transaction Render Server Decode Core Animation Draw Calls GPU Render Display 16.67 ms Core Animation Pipeline Application Core Animation Commit Transaction Render Server Decode Core Animation Draw Calls GPU Render Display Display 16.67 ms Core Animation Pipeline Application Core Animation Render Server Core Animation GPU Display 16.67 ms Core Animation Pipeline Application Core Animation Commit Transaction Render Server Core Animation GPU Display 16.67 ms Commit Transaction Commit Transaction Layout Set up the views Commit Transaction Layout Display Set up the views Draw the views Commit Transaction Layout Display Prepare Set up the views Draw the views Additional Core Animation work Commit Transaction Layout Display Prepare Commit Set up the views Draw the views Additional Core Animation work Package up layers and send them to render server Commit Transaction Layout Display Prepare Commit Layout Layout Display Prepare Commit layoutSubviews overrides are invoked View creation, addSubview: Populate content, database lookups Usually CPU bound or I/O bound Display Layout Display Prepare Commit Draw contents via drawRect: if it is overridden String drawing Usually CPU or memory bound Prepare Commit Layout Display Prepare Commit Image decoding Image conversion Commit Layout Display Prepare Commit Package up layers and send to render server Recursive Expensive if layer tree is complex Animation Three-stage process Application Render Server Animation Three-stage process Application Render Server 1. Create animation and update view hierarchy (animateWithDuration:animations:) Animation Three-stage process Application Render Server 2. Prepare and commit animation (layoutSubviews, drawRect:) 1. Create animation and update view hierarchy (animateWithDuration:animations:) Animation Three-stage process Application Render Server 2. PrepareLayout and commit animationDisplay (layoutSubviewsPrepare, drawRect:) Commit 1. Create animation and update view hierarchy (animateWithDuration:animations:) Animation Three-stage process Application Render Server 3. Render each frame 2. PrepareLayout and commit animationDisplay (layoutSubviewsPrepare, drawRect:) Commit 1. Create animation and update view hierarchy (animateWithDuration:animations:) Rendering Concepts Axel Wefers iOS Software Engineer Rendering Concepts Tile based rendering Render passes Example masking Tile Based Rendering Screen is split into tiles of NxN pixels Each tile fits into the SoC cache Geometry is split in tile buckets Rasterization can begin after all geometry is submitted Tile Based Rendering Screen is split into tiles of NxN pixels Each tile fits into the SoC cache Geometry is split in tile buckets Rasterization can begin after all geometry is submitted Tile Based Rendering Screen is split into tiles of NxN pixels Each tile fits into the SoC cache Geometry is split in tile buckets Rasterization can begin after all geometry is submitted Tile Based Rendering Screen is split into tiles of NxN pixels Each tile fits into the SoC cache Geometry is split in tile buckets Rasterization can begin after all geometry is submitted Tile Based Rendering Screen is split into tiles of NxN pixels Each tile fits into the SoC cache Geometry is split in tile buckets Rasterization can begin after all geometry is submitted Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL GPU Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL GPU Command Buffer Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL GPU Command Buffer Vertex Processing “Vertex Shader” Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL GPU Command Buffer Vertex Processing “Vertex Shader” Tiling Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL Tiler GPU Command Buffer Vertex Processing “Vertex Shader” Tiling Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL Tiler GPU Command Buffer Vertex Processing “Vertex Shader” Tiling Parameter Buffer Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL Tiler GPU Command Buffer Vertex Processing “Vertex Shader” Pixel Processing Tiling Parameter Buffer “Pixel Shader” Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL Tiler GPU Command Buffer Vertex Processing “Vertex Shader” Renderer Pixel Processing Tiling Parameter Buffer “Pixel Shader” Tile Based Rendering Rendering pass Render Server Application Core Animation Core Animation OpenGL Tiler GPU Command Buffer Vertex Processing “Vertex Shader” Renderer Pixel Processing Tiling Parameter Buffer Render Buffer “Pixel Shader” Masking Rendering passes Render Server Application Core Animation Core Animation OpenGL GPU Command Buffer Masking Rendering passes Render Server Application Core Animation Core Animation OpenGL Tiler Vertex Processing GPU Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render layer mask to texture Tiling Parameter Buffer “Pixel Shader” Mask Texture Masking Rendering passes Render Server Application Core Animation Core Animation OpenGL Tiler Vertex Processing GPU Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render layer mask to texture Tiling Parameter Buffer “Pixel Shader” Mask Texture Tiler Vertex Processing Pass 2 “Vertex Shader” Renderer Pixel Processing Render layer content to texture Tiling Parameter Buffer “Pixel Shader” Layer Texture Masking Rendering passes Render Server Application Core Animation Core Animation OpenGL Tiler Vertex Processing GPU Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render layer mask to texture Tiling Parameter Buffer “Pixel Shader” Mask Texture Tiler Vertex Processing Pass 2 “Vertex Shader” Renderer Pixel Processing Render layer content to texture Tiling Parameter Buffer “Pixel Shader” Layer Texture Tiler Vertex Processing Compositing pass “Vertex Shader” Renderer Apply mask to content texture Pixel Processing Tiling Parameter Buffer “Pixel Shader” Frame Buffer UIBlurEffect Axel Wefers iOS Software Engineer UIVisualEffectView with UIBlurEffect UIBlurEffect styles No effect Extra light Light Dark UIVisualEffectView with UIBlurEffect Rendering passes (best case) GPU Command Buffer UIVisualEffectView with UIBlurEffect Rendering passes (best case) Tiler GPU Vertex Processing Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render content Tiling Parameter Buffer “Pixel Shader” Content UIVisualEffectView with UIBlurEffect Rendering passes (best case) Tiler GPU Vertex Processing Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render content Tiling Parameter Buffer “Pixel Shader” Content Tiler Vertex Processing Pass 2 “Vertex Shader” Renderer Pixel Processing Downscaled Capture content Tiling Parameter Buffer “Pixel Shader” Content UIVisualEffectView with UIBlurEffect Rendering passes (best case) Tiler GPU Vertex Processing Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render content Tiling Parameter Buffer “Pixel Shader” Content Tiler Vertex Processing Pass 2 “Vertex Shader” Renderer Pixel Processing Downscaled Capture content Tiling Parameter Buffer “Pixel Shader” Content Tiler Vertex Processing Pass 3 “Vertex Shader” Renderer Pixel Processing Horizontal blur Tiling Parameter Buffer “Pixel Shader” Blur X Tiler Vertex Processing Pass 4 “Vertex Shader” Renderer Pixel Processing Vertical blur Tiling Parameter Buffer “Pixel Shader” Blur Y UIVisualEffectView with UIBlurEffect Rendering passes (best case) Tiler GPU Vertex Processing Command Buffer Pass 1 “Vertex Shader” Renderer Pixel Processing Render content Tiling Parameter Buffer “Pixel Shader” Content Tiler