Bringing GPU Acceleration to the Web Neil Trevett Khronos President NVIDIA Vice President Mobile Content © Copyright Khronos Group 2012 | Page 1 GPUs are Good at Many Things … Interactive ray tracing Physics simulation Gaming Battlefield 3, EA But traditionally GPUs have NOT been used for web and 2D graphics… Product design Data visualization … that’s about to change … © Copyright Khronos Group 2012 | Page 2 How Can GPUs enhance the Web? • More functionality - Helping to make HTML5 a complete apps platform - 3D graphics - WebGL - Compute - WebCL • More performance - Accelerating the bulk of web =2D - Accelerating key standards such as SVG and Canvas • Mobile computing changing the need for web acceleration Need MORE performance for TOUCH INTERACTIVITY AT LOWER POWER levels… © Copyright Khronos Group 2012 | Page 3 Mobile – a New Era in Computing 140 Cumulative Shipments 120 iOS & Android MacOS & Windows 100 80 Mobile industry is 60 20 Years faster to 100M/year shipments than PC 40 Units Millions in 20 0 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Source: Gartner, Apple, NVIDIA © Copyright Khronos Group 2012 | Page 4 Mobile Thermal Design Point 10” Screen takes 1-2W Resolution makes a difference! 7” Screen The iPad3 screen takes up to 8W 4-5” Screen takes takes 1W 250-500mW 2-4W 4-7W 6-10W 30-90W Typical max system power levels before thermal failure Even as battery technology improves - these thermal limits remain © Copyright Khronos Group 2012 | Page 5 How to Save Power? Write 32-bits to Memory 600pJ • Much more expensive to MOVE data than COMPUTE data Send 32-bits Off-chip • Energy efficiency must now be key metric 50pJ during silicon AND software design - Awareness of where data lives, where computation happens, how is it scheduled • Need to use hardware acceleration Send 32-bits 2mm 24pJ - Lots of processing in parallel - Efficient caching and memory usage - Reduces data movement 32-bit Float Operation For 40nm, 7pJ 1V process 32-bit Integer Add 1pJ 32-bit Register Write 0.5pJ © Copyright Khronos Group 2012 | Page 6 Needs for Accelerated Mobile Web In 5 years the number Power efficiency of pixels to process on by offloading mobile screens has from CPU to GPU GPU acceleration of the gone up by factor of complete web – including TWENTY … vector graphics High-resolution screens … and displays have have significantly more reached over 300DPI pixels to process 2048x1536 3100K Pixels 264 DPI Smooth 60Hz touch interactivity with ALL 720x1280 rich web content 1024x768 921K Pixels 786K Pixels 312 DPI 132 DPI 320x480 What open standards can help 153K Pixels 163 DPI accelerate the Web? Apple Apple HTC Apple iPhone iPad One X iPad3 © Copyright Khronos Group 2012 | Page 7 Khronos Connects Software to Silicon Khronos creates ROYALTY-FREE, OPEN STANDARD APIs for advanced hardware acceleration Low-level “Foundation” functionality at the software silicon interface needed on every platform Graphics, video, audio, compute, visual and sensor processing Shipping on billions of devices across multiple operating systems - rigorous conformance tests for cross-vendor consistency Khronos standards define the forward looking roadmap for the silicon community Khronos is OPEN for any company to join and participate Acceleration APIs BY the Industry FOR the Industry © Copyright Khronos Group 2012 | Page 8 API Standards Evolution WEB INTEROP, VISION MOBILE AND SENSORS DESKTOP OpenVL New API technology first evolves on high- Mobile is the new platform for Apps embrace mobility’s end platforms apps innovation. Mobile unique strengths and need Diverse platforms – mobile, TV, APIs unlock hardware and complex, interoperating APIs embedded – means HTML5 will conserve battery life with rich sensory inputs become increasingly important e.g. Augmented Reality as a universal app platform © Copyright Khronos Group 2012 | Page 9 3D API Family Tree ES3 is backward compatible Fixed function Programmable vertex so new features can be 3D Pipeline and fragment shaders added incrementally OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0 Content Content Content Mobile 3D WebGL 1.0 WebGL-Next OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0 ES-Next OpenGL ES 1.0 OpenGL 1.3 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL3.1 OpenGL3.3 OpenGL4.2 OpenGL 4.3 GL-Next OpenGL 3.0 OpenGL 3.2 OpenGL 4.0 OpenGL 4.1 OpenGL 4.3 is a Desktop 3D superset of DX11 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 © Copyright Khronos Group 2012 | Page 10 OpenGL ES Deployment in Mobile On PC – DirectX is used for Use of 3D APIs in Mobile Devices most apps. On mobile the Source: Jon Peddie Research situation is reversed OpenGL ES is the 3D API used in Android, iOS and almost every other mobile and embedded OS – other than Windows © Copyright Khronos Group 2012 | Page 11 WebGL – 3D on the Web – No Plug-in! • Leveraging HTML 5 and <canvas> element - WebGL defines JavaScript binding to OpenGL ES 2.0 - Enables a 3D context for the canvas • Low-level foundational Web API for accessing the GPU - Flexibility and direct GPU access - Enables higher-level frameworks and middleware JavaScript binding to OpenGL ES 2.0 Availability of OpenGL and Increasing JavaScript OpenGL ES on almost every performance. web-capable device HTML 5 Canvas Tag © Copyright Khronos Group 2012 | Page 12 WebGL Implementation Anatomy Content JavaScript, HTML, CSS, ... Content downloaded from the Web. Middleware can make WebGL accessible to non-expert 3D programmers JavaScript Middleware WebGL HTML5 Browser provides WebGL functionality alongside other HTML5 specs - no plug-in required JavaScript CSS OpenGL ES 2.0 OS Provided Drivers. WebGL on OpenGL Windows can use Google Angle to create DX9/Angle conformant OpenGL ES 2.0 over DX9 http://www.khronos.org/webgl/wiki/User_Contributions © Copyright Khronos Group 2012 | Page 13 WebGL Deployment • WebGL 1.0 Released at GDC March 2011 - Mozilla, Apple, Google and Opera working closely with GPU vendors • IE can be enabled with Chrome Frame - https://developers.google.com/chrome/chrome-frame/ • Mobile WebGL beginning to ship – Firefox, Opera, RIM - Pervasive mobile WebGL expected during next 12 months http://caniuse.com/#search=webgl WebGL is not enabled by default in desktop Safari. On iOS 5 WebGL is available to iAds © Copyright Khronos Group 2012 | Page 14 WebGL – Being Used by Millions Every Day © Copyright Khronos Group 2012 | Page 15 WebGL and Security • WebGL is Architecturally Secure - NO known WebGL security issues - Impossible to access out-of-bounds or uninitialized memory - Use of cross-origin images are blocked without permission through CORS - Browsers maintaining black lists - used if unavoidable GPU driver bugs discovered • DoS attacks and GPU hardening - Draw commands can run for a long time -> unresponsive system - Even without loops in shaders - WebGL working closely with GPU vendors to categorically fix this - Short term: mandate ARB_robustness and associated GPU watchdog timer - Longer term: GPUs need robust context switch and pre-emption WebGL is web-hardening GPU usage Helps ALL GPU accelerated portions of the browser stack © Copyright Khronos Group 2012 | Page 16 Why Khronos for WebGL? • Hardware API standards must take into account silicon design cycles - Multi-year pipeline of APIs that affect chips that take $100Ms to execute - Rigorous conformance tests and infrastructure • Khronos is unique forum where browser and GPU vendors can cooperate - Strong synergy from having both communities under one roof • Khronos is committed to being a good citizen in the larger Web community - Opened Khronos WebGL processes to enable cooperation with web community - http://www.khronos.org/webgl/public-mailing-list/ - http://www.khronos.org/registry/webgl/specs/latest/ - http://www.khronos.org/webgl/wiki/Testing/Conformance Khronos is the industry forum to drive hardware consensus and cooperation and advocate hardware support for higher-level software standards © Copyright Khronos Group 2012 | Page 17 OpenGL ES 3.0 Highlights • Better looking, faster performing games and apps – at lower power - Incorporates proven features from OpenGL 3.3 / 4.x - 32-bit integers and floats in shader programs - NPOT, 3D textures, depth textures, texture arrays - Multiple Render Targets for deferred rendering, Occlusion Queries - Instanced Rendering, Transform Feedback … • Make life better for the programmer - Tighter requirements for supported features to reduce implementation variability • Backward compatible with OpenGL ES 2.0 - OpenGL ES 2.0 apps continue to run unmodified • Standardized Texture Compression - #1 developer request! © Copyright Khronos Group 2012 | Page 18 Texture Compression is Key • Texture compression saves precious resources - Saves network bandwidth, device memory space AND memory bandwidth • Developers need the same texture compression EVERYWHERE - Otherwise apps need multiple copies of same texture for different platforms ASTC OpenGL ES 3.0 and OpenGL 4.3 Royalty-free ETC2 / EAC Extension -> Core BUT only optional in ES. MANDATED in once proven Quality Only 4bpp | 3 channel OpenGL ES 3.0 No alpha support OpenGL 4.3 Royalty-free NOT Royalty-free. Best quality. Platform Royalty-free Independent control of bit-rate Fragmentation ETC1 Backward compatible with ETC1 and # channels Mandated in ETC2: 4bpp | 3 channel 1 to 4 channel Android Froyo EAC: 4 (8)bpp | 1(2) channel COMBINE: 1-8bpp in fine steps DXTC/S3TC RGBA 8bpp | 4 channel (400M devices) Does not have 1-2 bit compression Windows WITH ALPHA PVRTC iOS Deployment 2008-2010 2012-2013 2014-> © Copyright Khronos