Freescale Semiconductor Document Number: AN4274 Application Note Rev. 0, 2/2011 High-End 3D Graphics with OpenGL ES 2.0 Three-dimensional (3D) graphics for embedded devices is Contents common in PDAs to tablets. The i.MX53 multimedia 1. Introduction . .1 2. 3D Graphics Primer . .3 processor offers a complete solution for the developers 3. EGL . .7 developing 3D graphics applications. 4. What is OpenGL ES . .13 5. OpenGL ES 2.0 Code Walkthrough . .19 This application note may be used as a startup guide to 6. Developing OpenGL ES 2.0 Applications With understand the principles of 3D graphics and the new Microsoft Visual Studio and WinCE 6.0 . .23 OpenGL ES 2.0 Application Programming Interface (API). 7. Conclusion . .26 8. References . .26 9. Revision History . .27 1 Introduction 3D graphics is a broad topic. This application note presents an overview of the 3D graphics and OpenGL ES 2.0 with its programmable pipeline. It also covers the 3D concepts needed for application development along with a basic example, which a developer can use as a template for further development. 1.1 What is OpenGL OpenGL is a widely adopted standard for real-time computer graphics, covering the most operating systems, and has thousands of applications. © Freescale Semiconductor, Inc., 2011. All rights reserved. Introduction OpenGL allows speed and innovation by incorporating a broad set of rendering, texture mapping, special effects, and other powerful visualization functions. Developers can leverage the power of OpenGL across all popular desktop and workstation platforms, ensuring wide application deployment. OpenGL is very stable, because it has been available for more than seven years on a wide variety of platforms, with a very high standard for additions, including backward compatibility, so the existing applications do not become obsolete. Also OpenGL is totally portable, allowing the developers to be sure that their applications will have the same visual result on any OpenGL API-compliant hardware, not worrying about the underlying operating system. Additionally, it allows the new hardware innovations to be implemented by the extensions mechanism, enabling the vendors to incorporate new features in their new products. OpenGL is well structured with an intuitive design and logical commands. Efficient OpenGL routines typically result in applications with fewer lines of code than those that make up programs generated using other graphics libraries or packages. In addition, OpenGL drivers encapsulate information about the underlying hardware, freeing the application developer from having to design for specific hardware features. Figure 1. The OpenGL Visualization Programming Pipeline OpenGL routines simplify the development of graphics software, from rendering a simple geometric point, line, or filled polygon to the creation of the most complex lighted and texture-mapped surfaces. OpenGL gives software developers access to geometric and image primitives, display lists, modeling transformations, lighting and texturing, anti-aliasing, blending, and many other features. All elements of the OpenGL state, even the contents of the texture memory and the frame buffer, can be obtained by an OpenGL application. OpenGL also supports visualization applications with 2D images treated as types of primitives that can be manipulated just like 3D geometric objects. High-End 3D Graphics with OpenGL ES 2.0, Rev. 0 2 Freescale Semiconductor 3D Graphics Primer 2 3D Graphics Primer Before going into further detail with OpenGL ES 2.0 example code, the concepts and terminology related to general 3D graphics need to be discussed. Some examples related to OpenGL ES will also be given, wherever possible. 2.1 Triangles and Lines Vertex is the most basic concept in OpenGL. It defines a “point” in a 3D space and has 3 coordinates, x, y, and z, usually defined as float values. Given this, a triangle is constructed by the union of three vertices (v1, v2, and v3). A line is constructed by the union of only two vertices. Figure 2. OpenGL and OpenGL ES 2.0 OpenGL can draw lines, triangles and quads, while OpenGL ES 2.0 can only draw and lines. 2.2 3D Meshes A 3D model or “Mesh” is a collection of triangles, which can be represented with an array. This is called as the vertex array, usually in the form of a triangle strip. High-End 3D Graphics with OpenGL ES 2.0, Rev. 0 Freescale Semiconductor 3 3D Graphics Primer Figure 3. 3D Mesh Model The 3D mesh in Figure 3 can be created with a 3D modeling tool, such as Blender, 3D Studio Max, Maya, and Lightwave 3D. A 3D mesh file can be exported in many formats, optimized or non-optimized. Below is an example of how an OBJ file looks (a universal 3D mesh format). #### # # OBJ File Generated by LightWave3D # LightWave3D OBJ Export v2.2 # #### # Object: 1 # # Vertices: 5 # Points: 1 # Lines: 0 # Faces: 6 # Materials: 1 High-End 3D Graphics with OpenGL ES 2.0, Rev. 0 4 Freescale Semiconductor 3D Graphics Primer # #### o 1 # Vertex list v -0.5 -0.5 0.5 v -0.5 -0.5 -0.5 v 0.5 -0.5 0.5 v 0.5 -0.5 -0.5 v 0 0.5 0 # Point/Line/Face list f 5 2 1 f 2 3 1 f 2 4 3 f 5 4 2 f 3 5 1 f 4 5 3 # End of file This file represents a pyramid. A pyramid has 5 vertices and 6 faces (triangles). The first part is the general information of the 3D mesh, which states the number of vertices and faces (triangles). The next part, vertex list, is as follows: # Vertex list v -0.5 -0.5 0.5 The “v” character stands for vertex. This line means that a vertex is constructed by the three x, y, and z coordinates (-0.5, -0.5, 0.5). The last part is about creating the actual triangles. # Point/Line/Face list f 5 2 1 High-End 3D Graphics with OpenGL ES 2.0, Rev. 0 Freescale Semiconductor 5 3D Graphics Primer The “f” character stands for “face” which actually means a triangle and the following three numbers are the vertex indexes. This means that a “face” is constructed by three vertices (the 5th, the 2nd, and the 1st). With all this information, any 3D model can be recreated from a “.OBJ” file. Though this file is not optimized at all, there are several options like using the Power VR (from Imagination Technologies) “.POD” file plug-in to export and load 3D meshes. This plug-in sorts the geometry (as triangle strips) to optimize them and avoid vertex redundancies. 2.3 Textures A texture is a 2D still image (such as bmp, jpeg, png, gif, and tga files). This image can be applied to a 3D surface (a triangle), by adding a pair of coordinates (U,V) to each vertex of the 3D mesh. After this, pixels are sampled from the image and interpolated (with the GPU) to achieve a “decal” effect on the 3D mesh. Figure 4. Textures In Figure 5, a jpg image (a human face) is being mapped into a 3D mesh. High-End 3D Graphics with OpenGL ES 2.0, Rev. 0 6 Freescale Semiconductor EGL Figure 5. 3D Mesh Example Texture coordinates are specified at each vertex of a given triangle. These coordinates are interpolated using an extended Bresenham’s line algorithm. If these texture coordinates are linearly interpolated across the screen, the result will be an affine texture mapping. This is a fast calculation, but there can be a noticeable discontinuity between adjacent triangles when these triangles are at an angle to the plane of the screen. The perspective correct texturing accounts for the vertices’ positions in 3D space, rather than simply interpolating a 2D triangle. This achieves the correct visual effect, but it is slower to calculate. Instead of interpolating the texture coordinates directly, the coordinates are divided by their depth (relative to the viewer), and the reciprocal of the depth value is also interpolated and used to recover the perspective-correct coordinate. This correction makes it such that in parts of the polygon that are closer to the viewer, the difference from pixel to pixel between texture coordinates is smaller (stretching the texture wider). In the parts that are farther, this difference is larger (compressing the texture). The GPU hardware implements perspective correct texturing, so, none of this has to be done by the CPU. 3EGL EGL handles graphics context management, surface/buffer binding, and rendering synchronization. It enables high-performance, and accelerated mixed-mode 2D and 3D rendering, using other Khronos APIs, such as OpenVG and OpenGL. EGL can be implemented on multiple operating systems (embedded Linux, WinCE, and Windows) and native window systems (such as X and Microsoft Windows). Implementations may also choose to allow rendering into specific types of EGL surfaces via other supported native rendering APIs, such as Xlib or GDI. EGL provides: High-End 3D Graphics with OpenGL ES 2.0, Rev. 0 Freescale Semiconductor 7 EGL • Mechanisms for creating rendering surfaces (windows, pbuffers, pixmaps) onto which client APIs can draw and share • Methods to create and manage graphics contexts for client APIs • Ways to synchronize drawing by client APIs as well as native platform rendering APIs To the extent possible, EGL itself is independent of definitions and concepts specific to any native window system or rendering API. However, there are a few places where native concepts must be mapped into EGL-specific concepts, including the definition of the display on which graphics are drawn, and the definition of native windows. Figure 6. EGL 3.1 How to Initialize EGL and Create a Rendering Context 3.1.1 Displays Most EGL calls include an EGLDisplay parameter.