J. lnf. Commun. Converg. Eng. 16(2): 125-129, Jun. 2018 Regular paper

A Simplified Graphics System Based on Direct Rendering Manager System

Nakhoon Baek* , Member, KIICE

School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Korea Abstract In the field of computer graphics, rendering speed is one of the most important factors. Contemporary rendering is performed using 3D graphics systems with windowing system support. Since typical graphics systems, including OpenGL and the DirectX library, focus on the variety of graphics rendering features, the rendering process itself consists of many complicated operations. In contrast, early computer systems used direct manipulation of computer graphics hardware, and achieved simple and efficient graphics handling operations. We suggest an alternative method of accelerated 2D and 3D graphics output, based on directly accessing modern GPU hardware using the direct rendering manager (DRM) system. On the basis of this DRM support, we exchange the graphics instructions and graphics data directly, and achieve better performance than full 3D graphics systems. We present a prototype system for providing a set of simple 2D and 3D graphics primitives. Experimental results and their screen shots are included.

Index Terms: Direct rendering manager, Efficient handling, Graphics acceleration, Light-weight implementation, Prototype system

I. INTRODUCTION Rendering speed is one of the most important factors for 3D graphics application programs. Typical present-day graph- After graphics output devices became publicly available, a ics programs need to be able to handle very large quantities large number of graphics applications were developed for a of graphics data. The larger the data size, and the more sen- broad spectrum of uses including computer animations, com- sitive to the rendering speed, the better the speed-up that can puter games, user experiences, and human-computer inter- be achieved, even for minor aspects of the graphics pipeline. faces. Production of the desired 2D or 3D images on the Although many graphics engines and full-development screen necessitated the development of graphics systems and tools are available, they all focus on efficient programming application program interface (API) libraries [1, 2]. interfaces rather than final execution speeds. Therefore, cur- A wide variety of graphics libraries were developed, rently, the OpenGL and DirectX graphics libraries are including OpenGL [3], DirectX [4], X Window systems [5], regarded as the most efficient approach to achieving high Display PostScript [6], Cairo [7], OpenInventor [8], and Qt speed precision rendering for large-scale data. [9]. The 3D graphics libraries are the contemporary norm in In contrast, graphics application programmers tend to computer graphics and related areas, with libraries like focus on both speed and the simplicity of the visualization OpenGL and DirectX in extensive public use. Typically, 3D process. To represent more realistic scenes, they need precise graphics application programs use the full scope of the fea- and accurate numerical graphics model data. Programmers tures offered by such 3D graphics libraries. tend to prioritize the easiest and most intuitive way of han-

Received 18 March 2018, Revised 14 June 2018, Accepted 14 June 2018 *Corresponding Author Nakhoon Baek (E-mail: [email protected], Tel: +82-53-950-6379) School of Computer Science and Engineering, Kyungpook National University, 80, Daehak-ro, Buk-gu, Daegu 41566, Korea.

https://doi.org/10.6109/jicce.2018.16.2.125 print ISSN: 2234-8255 online ISSN: 2234-8883

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by- nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Copyright ⓒ The Korea Institute of Information and Communication Engineering

125 J. lnf. Commun. Converg. Eng. 16(2): 125-129, Jun. 2018

dling these data, but one of the most important aspects in the development process is how to display these data on the screen efficiently and rapidly Therefore, in this paper, we present a low-level method for accelerated graphics rendering, designed especially for large- scale data visualization. We focus on the low-level data packets transferred between the main board and the graphics cards. After initialization of the rendering pipeline, we trans- fer the rendering data through these low-level data packets, achieving remarkable speed-up. More technically, our method is based on the direct ren- dering manager (DRM) system [10, 11]. We exclude the windowing system features from the 3D graphics architec- ture, and control the 3D graphics output directly with the DRM system.

II. OVERALL DESIGN OF OUR SYSTEM

In the early days of graphics systems development, graph- ics systems were controlled directly by user applications. Users could essentially handle the graphics hardware Fig. 1. The overall hierarchy of the full 3D graphics pipeline as implemented on Linux systems. directly, and the whole graphics system was relatively simple and efficient. As graphics systems were overhauled, the windowing sys- tem was introduced, and over time, full 3D graphics systems became an indispensable component of graphics rendering. Eventually, graphics systems became fully hierarchical; for example, the full 3D graphics system on typical Linux sys- tem has multiple layers, shown in Fig. 1. The start point of our idea is that modern windowing sys- tems have many overheads to handle the windowing system itself. Every graphics window needs to handle the user inter- actions and window-to-window events, and all other user interface system-dependent issues. In contrast, some com- puter graphics architectures adopt direct managing systems, Fig. 2. Windowing systems and direct management of framebuffers. so called because they access the framebuffer directly, as shown in Fig. 2. With Linux and its derived systems, the DRM module is used to access the framebuffers directly. In modern computer graphics architecture, the graphics processing unit (GPU) is essential to framebuffer management and various aspects of graphics processing. Thus, the modern DRM module now also manages the GPU in addition to the traditional framebuffer. From the architectural point of view, the Linux kernel con- tains a set of modules, and our focus is one of these, the DRM module. The GPU uses this module to provide its functions via predefined API function calls. The upper layers, including OpenGL and other applica- tion-level graphics libraries, use this DRM module as the standard way to transfer data to the GPU. Programmers can send and receive instructions and graphics data to and from the GPU, as shown in Fig. 3. Fig. 3. The role of the DRM module.

https://doi.org/10.6109/jicce.2018.16.2.125 126 A Simplified Graphics System Based on Direct Rendering Manager System

The DRM module provides some other, expanded, func- III. EXPERIMENTAL IMPLEMENTATIONS tionalities that carry out more advanced functions, including the kernel mode setting (KMS) module [12] and the graphics In the case of Linux kernels, the DRM module is used to execution manager (GEM) [13]. A detailed description of access the GPU. The upper layers, including OpenGL and these modules follows. other application-level graphics libraries, use this DRM module as the standard way to transfer data to the GPU [14]. A. Kernel Mode Setting Module Typical graphics programs send data as a mixture of the target data and the rendering commands for those data. For Most graphics displays, including LCD and CRT displays, large-scale data visualization, the target data portion is dra- usually have their own display configurations. For example, matically high, with very few rendering commands. Current an LCD display can show typical XGA resolution or 1,024 graphics libraries, however, use the conventional transfer by 768 pixels, as well as supporting a set of lower resolu- approach, transferring mixed sets of data and commands. tions. In our implementation, we use the standard OpenGL com- Additionally, the display can also provide some configura- mands for rendering, to facilitate easy and intuitive user tion alternatives. We can select the number of bits for the interfaces for graphics programmers. Then, we bypass the color, depth, and stencil configurations. These configurations high-level libraries, including OpenGL and similar others, are set by the low-level graphics hardware, and usually and instead send the DRM packets directly to the GPU. In called display modes. this way, we can remove the duplicated rendering commands In the Linux kernels, these display mode setting features typical of the rendering pipelines of OpenGL and other high- are isolated into another module, called the KMS module level graphics libraries. [12]. As a prototype implementation, we use a set of point clouds from LiDAR devices, which typically consist of more B. Graphics Execution Manager Module than 3 million color points, as shown in Fig. 4. With the original OpenGL programs, those 3 million data points are The GEM was designed to be the open-source Linux rendered with a set of rendering commands and data, subject graphics driver for handling in-kernel graphics memory man- to the normal OpenGL limitations. However, our implemen- agement, and performs this task, as well as handling graph- tation uses low-level DRM packets to send all the data to the ics at the system level. Therefore, it controls the graphics GPU, resulting in remarkable speed-up. memory and data bus to the GPU at the kernel level [13]. As shown in the screen shots of our prototype implementa- Full 3D graphics data usually have significantly large tion in Figs. 5 and 6, our system has no difficulty achieving graphics memory requirements, both in-kernel and out-of-kernel. typical 2D and 3D graphics outputs. Since we used our own Typical graphics programs should exchange the graphics 3D graphics pipeline implementation based on the DRM data between the CPU and GPU. Modern 3D graphics appli- library functions, our system does not use any existing com- cation programs usually require a lot of memory, especially mercially implemented 3D graphics library. for handling textures and the images. Our current implementation uses exactly the same syntax Most present-day Linux-based desktop PCs support the and calling conventions for the selected functions as the full spectrum of 3D graphics rendering features, and include GPU-based graphics hardware. Since the system should con- trol the GPUs to generate any images on the screen, the system should use the GEM module. From a low-level perspective, the user sends instructions and data to the GPU. From a high-level perspective, this results in an image being drawn on the screen. As such, the GEM module presents the sim- plest way of managing modern GPU-based graphics hard- ware. Another benefit of the GEM module is that a set of user application programs can share all the graphics resources through the GEM module. Legacy application programs were assigned with the exclusive right to the use of graphics resources, including the GPU-based hardware and/or soft- ware resources. In contrast, current GEM-based management enables user applications to all share the graphics resources, even without any conflicts. Fig. 4. An example of large-scale data visualization.

127 http://jicce.org J. lnf. Commun. Converg. Eng. 16(2): 125-129, Jun. 2018

Fig. 5. An example of 2D graphics output.

Fig. 7. An example of stencil test images.

Fig. 6. An example 3D graphics output.

OpenGL graphics standard. Thus, we avoid the need for Fig. 8. Textures and mipmapping test images. function specifications for our library functions. Additionally, our current implementation shows the source level compatibility with some OpenGL-based graphics appli- Table 1. Execution results (unit: frames per second) cation programs, as shown in Figs. 7 and 8. Notice that our Previous Our Acceleration current implementation only works in full-screen mode, implementation implementation ratio since it is based on the DRM module, which cannot support (A) (B) (B/A) any graphics windowing systems. Test_1 40.50 383.14 9.46 Execution results of our test programs are listed in Table Test_2 53.56 390.63 7.29 1. The regular OpenGL library was used in conjunction with Test_3 78.98 347.22 4.40 the X11 server system for comparison with our new imple- Test_4 71.29 510.20 7.16 mentation. Our system performs at least 4 to 7 times faster, meeting our expectations.

using lower-level DRM packets to achieve significant speed- IV. CONCLUSION up. Our prototype system also supports 2D and 3D graphics Typical present-day graphics tools and API libraries are processing, and the DRM-based features support depth-buf- designed to support simple and user-friendly user interfaces fers and stencil-buffers, as well as mipmap textures. Our cur- and function calls. In the case of large-scale data visualiza- rent implementation is also tested with typical OpenGL tion, rendering speed is more important. This paper outlines application program code-segments, and works correctly and a new way of efficiently rendering large-scale graphics data efficiently for the various test cases.

https://doi.org/10.6109/jicce.2018.16.2.125 128 A Simplified Graphics System Based on Direct Rendering Manager System

ACKNOWLEDGMENTS Herndon, VA: Mercury Learning & Information, 2016. [ 5 ] D. A. Young, The X Window System: Programming and Applications This research was supported by the Basic Science with Xt, 2nd ed. Englewood Cliffs, NJ: Prentice Hall, 1994. [ 6 ] Adobe Systems, Programming the Display Postscript System With X. Research Program through the National Research Foundation Reading, MA: Addison-Wesley, 1993. of Korea (NRF), funded by the Ministry of Education, Sci- [ 7 ] “Cairo: a 2D graphics library” 2014 [Internet], Available: http:// ence and Technology (No. 2016R1D1A3B03935488). www.cairographics.org/. This study was also supported by the BK21 Plus project [ 8 ] J. Wernecke, The Inventor Mentor: Programming Object-Oriented (SW Human Resource Development Program for Supporting 3D Graphics with Open Inventor. Reading, MA: Addison-Wesley, Smart Life) funded by the Ministry of Education, School of 1994. [ 9 ] G. Lazar, Mastering Qt 5: Create Stunning Cross-Platform Applica- Computer Science and Engineering, Kyungpook National tions. Birmingham, UK: Packt Publishing, 2017. University, Korea (No. 21A20131600005). [10] R. E. Faith, “The direct rendering manager: kernel support for the direct rendering infrastructure,” 1999 [Internet], Available: http://dri. sourceforge.net/doc/drm_low_level.html. REFERENCES [11] J. M. R. Fonseca, “Direct rendering infrastructure: architecture,” 2005 [Internet], Available: https://paginas.fe.up.pt/~mei04010/dri- architecture.pdf. [ 1 ] J. F. Hughes, A. Van Dam, J. D. Foley, M. McGuire, S. K. Feiner, D. [12] Archlinux, “Kernel mode setting,” [Internet], Available: https:// F. Sklar, and K. Akeley, Computer Graphics: Principles and wiki.archlinux.org/index.php/kernel_mode_setting. Practice, 3rd ed. Upper Saddle River, NJ: Addison-Wesley, 2014. [13] K. Packard and E. Anholt, “The Graphics Execution Manager: part [ 2 ] T. Capin, K. Pulli, and T. Akenine-Moller, “The state of the art in of the direct rendering manager,” 2008 [Internet], Available: https:// mobile graphics research,” IEEE Computer Graphics and Applications, lwn.net/Articles/283798/. vol. 28, no. 4, pp. 74-84, 2008. DOI: 10.1109/MCG.2008.83. [14] D. H. Youm and V. Kuraku, “Cost of content services to hybrid com- [ 3 ] M. Segal and K. Akeley, “The OpenGL graphics system: a specifi- puting,” Asia-Pacific Journal of Convergent Research Interchange, cation (version 4.5, core profile),” 2017 [Internet], Available: https:// vol. 2, no. 1, pp. 1-6, 2016. www.khronos.org/registry/OpenGL/specs/gl/glspec45.core.pdf. [ 4 ] F. D. Luna, Introduction to 3D Game Programming with DirectX 12.

Nakhoon Baek is currently a professor in the School of Computer Science and Engineering at Kyungpook National University, Korea. He received his B.A., M.S., and Ph.D. degrees in Computer Science from Korea Advanced Institute of Science and Technology (KAIST) in 1990, 1992, and 1997, respectively. His research interests include graphics standards, graphics algorithms and real-time rendering. He is now also the Chief Engineer of Dassomey.com Inc., Korea.

129 http://jicce.org