OpenGL ES update

Eisaku Ohbuchi Digital Media Professionals Inc. What is OpenGL ES?

• OpenGL for mobile devices – Gets rid of redundant / legacy features – Extensions to make it mobile-friendly – All the functionality of the desktop API • Versions – ES 1.0 / 1.1: fixed function – ES 2.0: -based via GLSL ES • A huge success! – The dominant native 3D API for mobile devices – Products from every major cell phone manufacturer – Moving rapidly into other embedded markets – >> 200M units shipped What’s New in 2009

• OpenGL ES 2.0 platforms are shipping! – Available in several popular handsets – Coming soon to set-top boxes, navigation devices, etc. – SDKs, books available Working Group Activities

• Conformance Tests – OpenGL ES 1.1 and 2.0 CTS upgrades released • Now preparing the next update with some bug fixes. • KTX file format – Standard file format for ETC1 compressed textures – Extended to handle all GL ES formats – Pixel data payloads can be passed directly to GL – Specification & sample loader code are coming very soon. • New technology – Extensions to ES 2.0 performance / functionality – Extensions to expose next-generation hardware features Improving GL / GL ES Interoperability

• The Dream – GL Everywhere OpenGL ES 1.x – Seamless portability

OpenGL • The Reality – Relationship between GL and ES is a bit messy – Porting code is more difficult OpenGL ES 2.0 than it needs to be – Keeping the in sync is becoming more difficult OpenGL Development Model The Model

Extensions GL Specs ES Specs

Cool New OpenGL ARB Tech ES WG

The Emerging Reality

GL ARB Specs Cool New Tech OpenGL ES Specs ES WG The ARB / ES Convergence TSG

• Introducing the ARB / ES Convergence TSG – Joint subcommittee of the ARB and the OpenGL ES WG • Charter – Encourage information sharing and communication – Produce joint technology and specification roadmap – Recommend ways to reduce existing differences over time – Recommend ways to minimize or avoid divergence in future • The Future – Could OpenGL ES become a profile of OpenGL? • Not a goal at present, but something to explore The Serialization TSG

• Serialization is a TSG under OpenGL ES – Just kicked off • Use cases – External displays, server and client come from different vendors – Communicating across processes – Better allocation of tasks with off-chip graphics HW – Tools for development, debugging, profiling – Networked gaming Questions? DMP solutions and SweetsUI engine

Eisaku Ohbuchi Digital Media Professionals Inc. Company overview

 Digital Media Professionals Inc.（DMP） www.dmprof.com  Location： Tokyo, JAPAN.  Incorporated: July/2002

 Products： – Graphics hardware IP cores – Training services – Software for our graphics hardware IPs DMP graphics hardware IPs

Advanced 3D Programmable graphics core graphics IP core （OpenGL ES 1.1 + (OpenGLES1.1, 2.0, DMP extension） OpenVG1.1)

PICA200 SMAPH-S

Tiny 3D graphics Vector graphics core IP core (OpenGL ES1.1) (OpenVG1.1)

PICA200 Lite SMAPH-F PICA200 for FPGA DMP Training services

 Location – DMP Inc. Seminar Room (2 minutes walk from JR Mitaka station)  Schedule  OpenGL ES programming training I January 14 - 15 2010  OpenGL ES programming training II January 21 - 22 2010  GLSL Programming training course (getting started) February 18 - 19 2010  See details - www.dmprof.com or http://www.khronos.org/news/events/ OpenGL ES programming training I

Curriculum Day 1 Day 2 Introduction Field of View OpenGL ES Overview Lighting(I) EGL Interface Blending Drawing Primitives Pixel Operation Setting Viewport (I) Transformation Fog Depth Buffer Frame Buffer Operation Drawing Scene Conclusions Goal Animation  To understand architecture of OpenGL ES and overview of some implementations.  To be able to read programs using OpenGL ES 1.1.  To be able to write 3D graphics applications for embedded systems with OpenGL ES 1.1. Capacity – 12 people Tuition Fee  49,560 Yen (taxes included, for 1 person, 2 days)  Khronos Member: 45,000 Yen (taxes included, for 1 person, 2 days) SIGGRAPH Asia開催記念キャンペーン! 特別価格 ¥37,800 OpenGL ES programming training II

Curriculum Day 1 Day 2 Introduction Dynamic Cube Mapping Lighting(II) Planer Reflection Buffer Object Dot3 Bump Mapping Point Sprite Extension Stencil Shadow Matrix Palette Extension Hint for Performance Evaluation Texture Mapping(II) Other Tips Compressed Texture Overview of OpenGL ES 2.0 Draw Texture Conclusions Goal  To be able to implement some visual effects using OpenGL ES 1.1 standard.  To be able to give consideration to performance aspects of OpenGL ES 1.1 application.  To understand basic concept of OpenGL ES 2.0 Capacity – 12 people Tuition Fee  49,560 Yen (taxes included, for 1 person, 2 days)  Khronos Member: 45,000 Yen (taxes included, for 1 person, 2 days) SIGGRAPH Asia開催記念キャンペーン! 特別価格 ¥37,800 GLSL Programming training course (getting started)

Curriculum Day 1 Day 2 Introduction Variables Chapter I: Overview of OpenGL ES 2.0 Functions OpenGL ES 2.0 Overview Vertex Shader Programmable Shader Overview Fragment Shader Chapter II: GLSL 1.1.0 Debugging and Tips for Development Basics of the Shader API Chapter III: GLSL ES 1.00 Mixed Mode OpenGL ES Shader Language Grammar Goal  To understand architecture of OpenGL ES 2.0.  To be able to read shader programs using GLSL 1.1.  To be able to write simple shader programs using GLSL 1.1. Capacity – 12 people Tuition Fee  99,750 Yen (taxes included, for 1 person, 2 days)  Khronos Member: 94,500 Yen (taxes included, for 1 person, 2 days) SIGGRAPH Asia開催記念キャンペーン! 特別価格 ¥76,650 Software for our graphics HW core – Sweets solution

• 3D graphics applications in embedded Customer application and systems tools – Expanding from 3D gaming to user interface applications SweetsUI Engine • DMP can provide a simple SW solution for demonstration and user interface Graphics engine development SDK OpenGLES1.1, 2.0 Driver SweetsUI engine. OS (Small and Wonder graphics Target HW Desktop emulator Engine for Embedded sysTemS.) SweetsUI 3D workflow overview

3D tools (Maya, Max, SoftimageXSI …)

User definition Lua script Target HW DMP App. or Engine PC emulator

Collada DMP Converter Sweets3D file Binary Sweets3D SweetsUI 3D features

Optimized 3D graphics engine for the . • C-based light weight graphics engine • OpenGL ES1.1 and DMP extension support • Script based control with Lua file – Run-time script execution support by DMP engine. • Collada file import – De-facto 3D authoring tool interchange format support • Optimized binary format export for Engine • Dynamic scene and animation control – Scene graph and animation manipulation using Lua scripting and C programming. Sweets 3D demo

• Simple UI application for photo exploration Questions? How to make an OpenVG application (not "Tiger")

Eisaku Ohbuchi Digital Media Professionals Inc. DMP graphics hardware IPs

Advanced 3D Programmable graphics core graphics IP core （OpenGL ES 1.1 + (OpenGLES1.1, 2.0, DMP extension） OpenVG1.1)

PICA200 SMAPH-S

Tiny 3D graphics Vector grapgics core IP core (OpenGL ES1.1) (OpenVG1.1)

PICA200 Lite SMAPH-F PICA200 for FPGA What is Tiger

• Major vector graphics sample – Khronos OpenVG reference implementation also includes this tiger sample. – Major functions of Tiger sample. • Path rendering • Antialiasing

• No bitmaps • No animations • No masks A not “Tiger” application - pdot_img

• Pseudo 3D vector graphics rendering – Functions • Path rendering • Bitmap • Masking • Font rendering • Rotating dots in 3D space • Select and preview interactions • You can try by downloading the source code from our web site: www.dmprof.com Overall structure: main()

main(){ // Parameter and variavble initialization .... // Init display Display if (!initDisplay(windowWidth, windowHeight, egldisplay, eglsurface)) { initialization cout << "Failed to initialize window" << endl; return 0; }

// Init app initApp();

// Display loop while (!quit) { // Key event handling Key event Main loop .... // Repaint repaint(); Repaint } 3D transformation void trans(int k) { float persX = dotData[k][0]; float persY = dotData[k][1]; float persZ = 150; • OpenVG 1.1 supports float xR,yR,zR;

transformation with 3x3 matrix. // rotation about x-axis: yR = persY * cos(rotX) - persZ * sin(rotX); • but it works in 2D space => No zR = persY * sin(rotX) + persZ * cos(rotX); persY = yR; persZ = zR; actual z value // rotation about y-axis: xR = persX * cos(rotY) + persZ * sin(rotY); • Here is pseudo 3D transformation zR = persZ * cos(rotY) - persX * sin(rotY); persX = xR; persZ = zR; for each dot. if (useRZ) { // rotation about z-axis: – Rotation about x-, y-, and z-axis xR = persX * cos(rotZ) - persY * sin(rotZ); yR = persY * cos(rotZ) + persX * sin(rotZ); – Perspective transformation persX = xR; persY = yR; }

float z = persZ + distZo; float souziK = (distScreen)/(z);

transData[k][0] = persX * souziK; transData[k][1] = persY * souziK; transData[k][2] = souziK; } Path rendering (drawing of stars)

float stUnit = strokeWidth; // starSegs[0] = VG_MOVE_TO_ABS; starSegs[1] = VG_MOVE_TO_REL; for(int k=2; k<6; k++) starSegs[k] = VG_LINE_TO_REL; starSegs[6] = VG_CLOSE_PATH; // starData[0] = orignX; starData[1] = orignY; starData[2] = -(stUnit / 2.0); starData[3] = stUnit / 6.0; starData[4] = stUnit; starData[5] = 0.0; starData[6] = -stUnit; starData[7] = -(2.0 * stUnit) / 3.0; starData[8] = stUnit / 2.0; starData[9] = stUnit; starData[10] = stUnit / 2.0; starData[11] = -stUnit;

Append // Append path data path data vgAppendPathData(myPath, 7, starSegs, starData); Path rendering (drawing of stars)

3. VG_LINE_TO_REL (stUnit, 0.0)

4. VG_LINE_TO_REL 2. VG_MOVE_TO_REL (orgX, orgY) (-stUnit, 2.0*stUnit/3.0) (-(stUnit / 2.0), stUnit / 6.0)

6. VG_LINE_TO_REL (stUnit/2.0, -stUnit) 5. VG_LINE_TO_REL (stUnit/2.0, stUnit) 7. VG_CLOSE_PATH

1. VG_MOVE_TO_ABS (orgX, orgY) Path rendering (drawing of stars)

Append // Append path data path data vgAppendPathData(myPath, 7, starSegs, starData); // Transform matrix generation vgSeti(VG_MATRIX_MODE, VG_MATRIX_FILL_PAINT_TO_USER); Transform vgLoadIdentity(); vgTranslate(tmpData[0]-(zsc*swo2), tmpData[1]-(zsc*swo2)); matrix vgScale(zsc*swoD, zsc*swoD); // Blend mode configuration Blending vgSeti(VG_BLEND_MODE, spec ? VG_BLEND_ADDITIVE : mode VG_BLEND_SRC_OVER); Paint // Paint pattern set pattern vgPaintPattern(imgPaint, img[imgIndx]); // Draw path Draw path vgDrawPath(myPath, fShape ? VG_FILL_PATH : VG_STROKE_PATH); vgClearPath(myPath, VG_PATH_CAPABILITY_ALL); Draw image, blending and masking

Set masking int imgIndx = (5 + vOffs) % numImg; and blend vgSeti(VG_MASKING, VG_TRUE); mode vgSeti(VG_BLEND_MODE, VG_BLEND_SRC_OVER);

float fAlpha = (float)fCount/(float)fStep; float ctVal_w[8] = { 1.0,1.0,1.0,(1.0-fAlpha), 0.0,0.0,0.0,0.0}; Color vgSetfv(VG_COLOR_TRANSFORM_VALUES, 8, ctVal_w); transform vgDrawImage(img[(imgIndx==0) ? (numImg-1) : (imgIndx-1)]); ctVal_w[3] = fAlpha; vgSetfv(VG_COLOR_TRANSFORM_VALUES, 8, ctVal_w); Draw vgDrawImage(img[imgIndx]); image if (++fCount==fStep) fading = false; Color transformation Benefit of vector graphics

• Scalable, high quality rendering with lightweight graphics core – Font rendering – 2D or 2.5D user interface – Map rendering • Native OpenVG application can be fast ! – Few overhead – Seamless programming with native application. • Let’s develop graphics applications with OpenVG API ! Questions?