GPU Benchmarking - for Handsets from a UI Perspective
Total Page:16
File Type:pdf, Size:1020Kb
GPU Benchmarking - for handsets from a UI perspective ERIK BÄCKSTRÖM KRISTIAN SYLWANDER Master’s Thesis at LTH Supervisor: Fredrik Ademar Examiner: Micheal Doggett Abstract The graphical processing unit (GPU) is becoming an essential part of mo- bile phones and handsets. This improves the ability of handling graphics in handsets significantly. With this new chip it becomes possible to develop new stunning games but also new innovative 3D GUI (Graphical User Interfaces). Today there exist a couple of benchmarking programs targeting mobile GPUs but all of these benchmarking tools are all focused on game performance and not GUI performance. The aim with this master’s thesis is to develop a bench- marking program for mobile GPUs focusing on GUI performance. We have identified five features that are important when benchmarking a handset, from a GUI perspective: scissoring, texture uploads, alpha blending, load times and frame rate. Additionally, we have benchmarked five different devices using our tool. From the test we could conclude that there were big differences in the performance compared to the specification sheets. Acknowledgement This master thesis is done as a completing part of our education at LTH (Lund Institute of Technology) with focus on computer science. During the course of our master thesis we have received invaluable supervision and help from a number of persons both at TAT and at LTH. Therefore we would like to take the opportunity to thank these persons for their advise, knowledge and dedi- cation. First we would like to thank our supervisors at TAT Fredrik Ademar, Fredrik Berglund and Dan Gärdenfors. We also want to thank Michael Dog- gett, our examiner at LTH. Special thanks are directed to Michael Winberg and Emil Wassberger for their invaluable help. We would also like to thank our girlfriends Sania and Pernilla, for all their love and support, and for their comments on the thesis. Contents 1 Introduction 1 1.1 The Problem . 1 1.2 Our Aim . 2 1.3 Collaboration . 2 2 Background 3 2.1 OpenGL . 3 2.1.1 OpenGL 1 . 3 2.1.2 OpenGL 2 . 4 2.2 OpenGL ES . 5 2.3 EGL . 5 3 Benchmarking a GPU 7 3.1 Current Tools on the Market . 7 3.1.1 GLBenchmark . 7 3.1.2 JBenchmark . 8 3.1.3 Futuremark 3D Mobile . 9 3.2 Why a new Benchmarking tool? . 10 4 Method 11 5 Result 13 5.1 Test Framework . 13 5.1.1 PolluxGL . 13 5.2 GUI Characteristics . 16 5.3 Test Cases . 16 5.3.1 Basic . 17 5.3.2 Fill Rates . 17 5.3.3 Textures . 18 5.3.4 Frame Buffer Objects . 18 5.3.5 Triangle Throughput . 19 5.3.6 Shaders . 19 5.3.7 GUI Specific . 19 5.3.8 Pixel Pipeline Performance . 20 5.3.9 Optimizations . 21 5.4 Visualize Result . 21 6 Benchmarked Devices 23 6.1 Tested Devices . 23 6.2 Benchmarking Result . 23 6.2.1 Basic . 24 6.2.2 Fill rates . 24 6.2.3 Textures . 24 6.2.4 Frame Buffer Objects . 25 6.2.5 Triangle Throughput . 25 6.2.6 Shaders . 25 6.2.7 GUI Specific . 26 7 Discussion 27 7.1 Incorporation in a development process . 28 8 Conclusion 29 Appendices 29 A Test Case Descriptions 31 A.1 Frame Buffer Clear . 31 A.2 Draw Triangle . 31 A.3 Color Fill . 31 A.4 Z Fill . 32 A.5 Color-Z Fill . 32 A.6 Texel Rate . 33 A.7 Texture Upload Rate . 33 A.8 Texture Upload Format . 33 A.9 Alpha Blending . 33 A.10 Scene Alpha . 34 A.11 Scissoring . 34 A.12 Frame Buffer Object (FBO) . 34 A.13 Frame Buffer Count . 35 A.14 Triangle Count . 35 A.15 Triangle Count VBO . 35 A.16 Vertex Buffer Object (VBO) count . 35 A.17 Photo River . 36 A.18 Photo River with Mip-map . 36 A.19 Shader Program Switch . 36 A.20 Shader Precision . 37 A.21 Shader Limitation . 37 A.22 Subtexture updates . 37 A.23 Bump Mapping . 37 A.24 Timer test . 38 A.25 Non power of two textures . 38 A.26 Pixel Pipeline Performance . 38 Bibliography 41 Chapter 1 Introduction In the first quarter of 2009 mobile phone sales dropped 8.6 percent compared to the first quarter of 2008. During the same period Smartphone sales increased with 12.7 percent [1]. One reason for this increase is the introduction of the iPhone, which has started the trend of touch screens and put focus on graphics and usability in handsets [2]. Now vendors are trying to create the most visually appealing graphical user interfaces (GUIs) to meet the competition from Apple. But to be able to create this more advanced graphics, new hard- ware is needed in the phones. The key component is the graphical processing unit (GPU). It enables handsets to render advanced 3D graphics, mostly used for 3D games. Since the mobile game industry is estimated to generate 10 million dollars this year it is not supris- ing that the focus is on games, but the GPU can also be used to create new innovative 3D GUIs, and since the user experience is becoming one of the main selling points for mobile phones this is a great opportunity for the operators and vendors [3]. Mobile manufactur- ers need to have the latest technology in their phones to be able to compete on the global market, but they also need to cut their costs. Different mobile phones therefore perform very differently even though they support the same standards. The graphics standard is not always followed and sometimes functions are poorly implemented which can lead to un- expected behavior and/or bad performance especially on a prototype device[4]. Graphics application programming interfaces (API) are often implemented with the main purpose of running games which in turn can lead to functions not commonly used in games being im- plemented incorrectly or inefficiently due to short time frames during development. These functions can for instance be useful in a GUI[4] 1.1 The Problem The problem is that the benchmarking tools available today focus on game performance which makes them unsuitable for GUI benchmarking. Another aspect is that when de- veloping a GUI or anything else using graphics one would like to have information about which specific features that are fast/slow on a certain device. That information is, to our knowledge, not given by any of the tools available today as these use scenario-based test cases. Due to the lack of a sufficient 3D GUI benchmarking tool another problem arises. 1 CHAPTER 1. INTRODUCTION When an unfamiliar device is used in a development process intended to create a graphics application of some kind, the device is not benchmarked prior to the start of the project. This will most certainly lead to surprises about the performance which will arise during development which in turn can lead to costly delays. 1.2 Our Aim Our aim is to develop a benchmarking tool with the purpose of finding the limitations and bottlenecks of a mobile GPU. The tool will be designed with the development of a 3D GUI in mind. Hence the tests performed by the program will be focusing on GUI specific fea- tures. It will also focus on individual low-level features in order to find minor bottlenecks caused by these features. We will also investigate the possibility of using this program to reduce any delays caused by insufficient knowledge about the device from occurring in the development process. 1.3 Collaboration This master thesis has been carried out in collaboration with The Astonishing Tribe (TAT) and the Department for Computer Science at the Faculty of Engineering at Lund University. TAT is a company that develops solutions for GUIs in mobile devices. They have, among many other things, developed the GUI for Android (Google’s operating system for mobile devices) and their products can be found in over 300 million devices worldwide [5]. 2 Chapter 2 Background A very important factor that made it possible to run advanced 3D games on mobile de- vices is the introduction of Open Graphics Library for Embedded Systems (OpenGL ES or GLES), a platform independent API for rendering 2D and 3D graphics on devices with limited resources. In this section we will give a brief background to this API as well as an introduction to the benchmarking tools available on the market today. 2.1 OpenGL Open Graphics Library (OpenGL) is a cross-platform API for rendering 2D and 3D com- puter graphics [6] . In other words, OpenGL is a set of commands for the creation of geometric objects and commands that control how these objects are rendered. One of the strengths of OpenGL is the possibility to develop extensions to add new features that are not specified by the OpenGL standard, which makes sure that OpenGL is current with the latest graphics and rendering algorithms available. Furthermore, it is important to let program- mers adapt their applications to different hardware to achieve the goal of a cross-platform API. OpenGL was initially developed by Silicon Graphics Inc (SGI) as an initiative to create a single, vendor-independent graphics API [7]. However, when the Architecture Re- view Board (ARB) was founded in 1992 they took over maintenance of OpenGL until they became a part of the Khronos Group in 2006. 2.1.1 OpenGL 1 The first version (1.0) of OpenGL was released in 1992 [6]. It contained features like per- vertex lighting, texturing and blending . When OpenGL Version 1.1 was released later that year vertex arrays were introduced as one of the more important additions.