COVER STORY AIGLX

AIGLX and the rise of the desktop GRAPHICGRAPHIC HORIZONSHORIZONS

Red Hat’s head of X development describes the evolution of AIGLX. www.sxc.hu BY KEVIN E. MARTIN

ed Hat developers have been display device, while the graphics card from the window to the icon in the working on an alternative to , hardware has shifted dramatically to em- Gnome panel to show the transition. Rwhich they have dubbed AIGLX. phasize 3D. Integrating 3D into the desk- AIGLX promises hardware accelerated top has long been the goal, but until re- Building on the Past desktop effects, while at the same time cently, it has not been possible. Other The current vision for an OpenGL-based facilitating integration with the Xorg in- operating systems have also recognized composited desktop is built on several frastructure. this paradigm shift – Apple is using key technologies that have been devel- The Fedora Rendering Project de- OpenGL through its Quartz [1] composi- oped over the past several years. scribes AIGLX as “a project that aims to tor architecture and Sun has a research Throughout most of the 1990s, the enable GL-accelerated effects on the project called Looking Glass [2] to exper- only open source implementation of standard desktop. They add, “We have iment with using Java3D on their desk- OpenGL was Mesa [3], which was, at a lightly modified X server (that includes top. that time, a software-only client-side li- a couple of extensions), an updated A second problematic area is that all brary that implemented the OpenGL in- Mesa package that adds some new pro- of our drawing is rendered directly into terface. Then, in late 1998, Precision In- tocol support, and a version of the front buffer. What this means is that sight began developing the Direct Ren- with a composite manager. The end re- users can see rendering artifacts while dering Infrastructure (DRI) [4]. This sult is that you can use GL effects on the desktop scene is being constructed. project brought hardware-accelerated 3D your desktop with very few changes, you Most drawing is very fast, so it usually graphics to Linux, but as the name im- can turn it on and off at will, and you appears as a visually displeasing blur be- plies, it was only for direct rendering ap- don’t have to replace your X server in fore the final image is visible, but some- plications. Indirect rendering was imple- the process.” times is it much worse and you can see mented using the software Mesa code. AIGLX is part of an ecosystem of individual elements being drawn. Tradi- The DRI allowed applications that graphics components that interact to de- tionally, toolkits have had to work wanted to take advantage of the 3D liver better performance and more vivid around this problem by drawing directly hardware to do so through the OpenGL images. How did all this happen and to host-memory pixmaps and then copy- library (libGL). Traditionally, only what does it mean for the user? This ar- ing the finished image to the screen. specialized 3D applications wrote ticle describes the context for the evolu- The third problem is the static nature directly to OpenGL or one of its toolkits. tion of AIGLX and a new generation of of the windowing states, and the fact These apps had no (or very little) direct graphics technologies. that the transitions between those states knowledge of the 3D hardware under- are either instantaneous or have very lying libGL because the DRI itself is a The State of Computer primitive transition animations. For ex- general purpose infrastructure that loads Graphics ample, minimizing or unminimizing a card-specific driver to handle all The current desktop design is starting to windows either simply pops windows accelerated OpenGL functions. The card- show its age in several areas. First, the into or out of existence, or very simple specific drivers have a well-defined set current desktop is designed around a 2D window outlines are drawn in sequence of entry points to handle initialization

28 ISSUE 68 JULY 2006 WWW.LINUX - MAGAZINE.COM AIGLX COVER STORY

and hook themselves directly into the sitions. For example, we created the dering using the same card-specific libGL library. wobbly window effect, where windows driver code that is loaded on the client- With the relatively recent development were modeled with a simple spring sys- side by libGL; however, it was not a sim- of the Composite extension [5], we now tem, so that dragging a window around ple task, and the driving issue to make have the ability to redirect 2D pixel data would distort it as if you were pulling on this happen did not occur until we came into host-memory or off-screen pixmaps. one of the springs. to the GL-based composited desktop. This pixel data can then be copied to the The software Mesa driver used in the display buffer as needed to update what The New Gl-based initial DRI work was based on the client- the user sees on their screen as their Composite Desktop side version of Mesa, which translated desktop. This effectively gives us the One of the primary performance prob- OpenGL requests into X11 drawing com- ability to double-buffer 2D data, an abil- lems with Luminocity was the number mands. We modified this code, which ity that has long been used by OpenGL of data copies required to get the redi- was previously called libX11, to instead applications to eliminate visual artifacts. rected window pixel data into a texture call the equivalent internal X server This ability is not new. The double- that could be used by the hardware. functions. We named this version GL- buffer extension (DBE) allowed individ- Ultimately, we want to get to the point core. The interfaces used to initialize ual apps to double buffer their output. where no copying of redirected window and call into GLcore were the __GLinter- What makes Composite unique is that pixel data is necessary, i.e., a redirected face and __GLdrawablePrivate structs, individual application do not need to window could be drawn to an off-screen which are part of the OpenGL sample have direct knowledge of the DBE. In- pixmap in the correct format to be used implementation (SI) [7]. stead, an external application known as directly by the hardware’s 3D engine. To However, the DRI project used Mesa the composite manager controls when this end, we implemented two new in- instead of the SI, so the interface to the windows are redirected and how the frastructure-level technologies: Acceler- card-specific drivers were Mesa-based, pixel data is copied to the display buffer. ated Indirect GLX (AIGLX) and the GLX which is quite different from the GLcore EXT_texture_from_pixmap extension interface. This impedance mismatch was Luminocity (TFP). On top of this new infrastructure, one of the reasons that it took so long to In late 2004, we began experimenting we built a new scene-graph-based com- implement AIGLX. So, to take advantage with using OpenGL to render redirected positing library that is used by the of the DRI/ Mesa card-specific drivers windows in our Luminocity project [6]. Metacity window manager to implement within the X server for AIGLX, we had to The example composite managers that the OpenGL animation effects. reconcile these two interfaces. had been developed previously used the We developed a new abstraction layer Render extension to copy the window Accelerated Indirect [8], which is heavily based on the DRI data to the screen (e.g., xcompmgr). As Rendering interface, to provide the glue logic be- noted earlier, Apple was using OpenGL As noted earlier, indirect rendering was tween the server-side GLX extension to achieve similar effects, and Sun had completely unaccelerated in the initial code and the card-specific driver. The been experimenting with using Java3D DRI project. The plan had always been new interface provides three objects: in their Looking Glass project. to implement accelerated indirect ren- __GLXscreen, __GLXcontext and The basic idea behind Luminocity was to create OpenGL textures using the pixel data from each redirected window and then draw textured rectangles to the front buffer using each of those textures. Since the only open source hardware-ac- celerated OpenGL available at that time was through the DRI, Luminocity was developed to use direct rendering. The problem with this approach was that the pixel data had to be copied from the X server to the client before it could be used as a texture. These extra copy steps hurt performance. Another difference was that previous composite managers were separated from the window manager, whereas Lu- minocity was a combined composite and window manager. By combining the two, Luminocity could not only copy window data to the screen and render static effects like window shadows, but it could also animate various state tran- Figure 1: Luminocity supports transparent windows.

WWW.LINUX - MAGAZINE.COM ISSUE 68 JULY 2006 29 COVER STORY AIGLX

__GLXdrawable. Methods for allocating A better approach is to keep the pix- It should be noted that, while AIGLX the DRI-specific objects and calling into map data in the X server address space and TFP are critical to our GL-based the card-specific driver are contained en- where it was rendered and use it directly composited desktop, we have developed tirely within the abstraction layer, which as the source for a texture operation. them so they can be used independently we called the DRI provider. GLX_EXT_texture_from_pixmap pro- by application developers. For example, Since not all graphics cards have card- vides the interface to make that happen. [9] is another window/ compos- specific 3D drivers, and since several As noted above, the ideal solution is to ite manager that takes a different ap- other servers (e.g., Xnest) that provide have the graphics card render the win- proach but works well using technolo- GLX support cannot use hardware driv- dow contents into an off-screen buffer, gies we have developed [10]. ers, the GLcore module must remain which would then be used directly (i.e., available, so we rewrote the top level of with no copying or conversion) as the Technology Preview GLcore to use the new interface. This al- input to the hardware texture engine. To We currently have a technology preview lows it to be used in place of the card- implement this solution, we will need that redirects windows to host-memory specific drivers when needed or desired, additional infrastructure work (e.g., pixmaps in the X server and then uses and it is called the GLcore provider. memory management) as well as addi- the texture from pixmap extension to use To initialize the GL module for each tional card-specific driver work, both of the host-memory pixmap as a texture screen, a stack of GL providers are called which are currently in development. source. This eliminates all but the last and the first provider that returns a non- Our intermediate TFP solution redi- data copy and provides reasonable per- NULL. __GLXscreen claims that screen. rects window data into host-memory formance. You'll find more information, This mechanism allows for future GL pixmaps and calls the texture operations demos, and status about our GL-based modules to implement their own __GLX- directly through the new AIGLX abstrac- composited desktop project at [11]. provider and hook into the provider tion layer interface to the Mesa/ DRI There is still much to do. We and oth- stack. card-specific driver. By rendering directly ers in the open source development to host-memory pixmaps, we bypass the community are in the process of adding GLX_EXT_texture_from_ “read from framebuffer” operation, several new technologies. These include pixmap which can be quite slow. input transformation, advanced memory With accelerated indirect OpenGL, we management, redirecting extensions can now render directly from within the Metacity (e.g., Xv, GL, DRI), frame buffer objects, X server process; however, we still need Luminocity was a toy window and com- FBconfigs, and full GLX 1.3 support. As to be able to use the window pixel data posite manager that allowed us to rap- these and other technologies are devel- that was redirected to a pixmap with the idly prototype various technologies and oped, we will continue to update our Composite extension as a texture. This is experiment with using OpenGL in a GL-based composited desktop solution what the texture from pixmap GLX ex- composited desktop. We could have ex- to take advantage of the new features. ■ tension provides. panded Luminocity into a fully func- The simple approach would be to copy tional window manager, but this would INFO the data either through the protocol via have involved recreating the years of [1] Quartz compositor architecture: XGetImage or through a shared-memory work that went into our standard desk- http:// www. apple. com/ macosx/ pixmap into the client’s address space, top window manager, Metacity. Instead, features/ quartzextreme and then the direct-rendered composite we took what we learned from Luminoc- [2] Looking glass desktop: http:// www. manager could use that data as the ity and reimplemented it in Metacity. sun. com/ software/ looking_glass source for a glTexImage2D or glDrawPix- Our approach was to create a new [3] Mesa: http:// www. mesa3d. org els call. This does not work in practice OpenGL scene-graph-based compositing [4] DRI: http:// dri. freedesktop. org/ wiki due to the high overhead of copying library, called libcm, that would encap- [5] Composite extension: pixel data to and from video memory. sulate the methods used by the rest of http:// cvs. freedesktop. org/ xlibs/ the window manager to draw the desk- CompositeExt/ protocol?view=markup Kevin E. Martin has been working top. Metacity could then hook various [6] Luminocity: on the for the state transition animations into the http:// live. gnome. org/ Luminocity past 17 years. He was the principle scene-graph as needed. [7] OpenGL sample implementation: architect of the Direct Rendering In- http:// oss. sgi. com/ projects/ ogl-sample By making the full OpenGL interface frastructure (DRI) and Distributed available, we can create arbitrarily com- [8] GLX abstraction layer: Multihead X (DMX) system. He has http:// lists. freedesktop. org/ archives/ also developed many 2D and 3D plex animations that are only limited by xorg/ 2006-February/ 013326. html what we can dream and what the hard- drivers. He is the head of X and [9] Compiz: OpenGL development at Red Hat, ware is capable of. Some common ef- http:// en. opensuse. org/ Compiz serves on the X.Org Foundation fects that we already have or are in the [10] Compiz with AIGLX:

THEAUTHOR Board of Directors, chairs the X.Org process of implementing include minimi- http:// lists. freedesktop. org/ archives/ Foundation’s Modularization Work- zation, maximization, menu fade in/ out, xorg/ 2006-March/ 013577. html ing Group and was the overall re- drop shadows, window transparency, [11] Fedora rendering project: lease manager for the last Xorg re- and workspace switching. Many others http:// fedoraproject. org/ wiki/ leases. will be developed as the need arises. RenderingProject/ aiglx

30 ISSUE 68 JULY 2006 WWW.LINUX - MAGAZINE.COM