Hacking on 3D Eleni Maria Stea Introduction to Mesa 3D/OpenGL The most widely used source OpenGL/Vulkan implementation

Eleni Maria Stea Hacking on Mesa 3D 2 / 26 Introduction to Mesa 3D/OpenGL Historical

▶ It started as an open source (MIT license) software OpenGL implementation. • swrast: software rasterizer

▶ Then some hardware drivers were added as backends. • , AMD , , VMware virtual GPU

▶ More , specifications were implemented. • Vulkan • OpenGL ES1, ES2 & ES3 • OpenCL, OpenVG, VDPAU, XVMC, EGL interface

▶ More software drivers were written. • llvmpipe, softpipe

Eleni Maria Stea Hacking on Mesa 3D 3 / 26 Introduction to Mesa 3D/OpenGL

Mesa 3d is a big project :-) We are going to focus on the OpenGL development.

Eleni Maria Stea Hacking on Mesa 3D 4 / 26 Introduction to Mesa 3D/OpenGL The OpenGL API as part of the Graphics Stack

Indirect Rendering

GLX (OpenGL extension to the )

Direct Rendering

DRI (Direct Rendering Infrastructure)

DRM ()

A combination of the two

https://en.wikipedia.org/wiki/Mesa_(computer_graphics) GLX + AIGLX (Accelerated Indirect GLX)

Eleni Maria Stea Hacking on Mesa 3D 5 / 26 Developing the OpenGL So, what’s our contribution to the OpenGL implementation?

▶ we extensions (16-bits etc) ▶ we add new features for the extensions to work ▶ testing and bug fixes ▶ releases

This means that we occasionally make changes to: ▶ the mesa core ▶ the glsl compiler ▶ the hardware drivers ▶ and anywhere else needed

Eleni Maria Stea Hacking on Mesa 3D 6 / 26 Developing the OpenGL OpenGL Extensions 1/4 : The concept behind

▶ OpenGL is extensible by design.

▶ Any hardware manufacturer who wishes to add a new feature that was missing from OpenGL can write an extension to provide it.

▶ Any user wishing to use the feature can then use the extension through the extension mechanism.

▶ (that releases the OpenGL spec) decides which extensions will become part of the next OpenGL core version.

▶ The manufacturers that write their own OpenGL implementations (, Apple, Intel/Mesa, AMD/Mesa etc) and wish to release that new version must implement all the extensions that have been decided to be part of the core.

Eleni Maria Stea Hacking on Mesa 3D 7 / 26 Developing the OpenGL OpenGL Extensions 2/4: Τhe specs

OpenGL Registry clone https://github.com/KhronosGroup/OpenGL-Registry.git

Extensions Specifications

▶ overview, contributors etc ▶ dependencies ▶ errors ▶ detailed descriptions

Eleni Maria Stea Hacking on Mesa 3D 8 / 26 Developing the OpenGL OpenGL Extensions 3/4: Required changes

Adding a new extension requires changes in the API, the hardware drivers code (see i965), to the GLSL compiler and elsewhere.

$ ack-grep -i -l ARB_shader_storage_buffer_object mesa

include/GL/glcorearb.h mesa/src/mapi/glapi/registry/gl.xml include/GL/glext.h mesa/src/mapi/glapi/gen/.build src/mapi/glapi/registry/gl.xml mesa/src/mapi/glapi/gen/Makefile.am src/mapi/glapi/gen/Makefile.am mesa/src/mapi/glapi/gen/Makefile src/mapi/glapi/gen/gl_API.xml src/mesa/drivers/dri/i965/intel_extensions. mesa/src/mapi/glapi/gen/Makefile.in src/mesa/main/extensions_table.h mesa/src/mapi/glapi/gen/gl_API.xml src/mesa/main/context.c mesa/src/mesa/drivers/dri/i965/intel_extensions.c src/mesa/main/mtypes.h mesa/src/mesa/main/extensions_table.h mesa/src/mesa/main/context.c src/mesa/main/bufferobj.c src/mesa/main/get.c mesa/src/mesa/main/mtypes.h src/mesa/main/version.c mesa/src/mesa/main/bufferobj.c src/mesa/main/get_hash_params.py mesa/src/mesa/main/get.c mesa/src/mesa/main/version.c src/mesa/main/uniforms.c mesa/src/mesa/main/get_hash_params.py src/mesa/state_tracker/st_extensions.c src/compiler/glsl/glsl_lexer.ll mesa/src/mesa/main/uniforms.c src/compiler/glsl/ir.h mesa/src/mesa/state_tracker/st_extensions.c mesa/src/compiler/glsl/glsl_lexer.ll src/compiler/glsl/ast_function.cpp mesa/src/compiler/glsl/ir.h src/compiler/glsl/glsl_parser_extras.cpp src/compiler/glsl/glsl_parser_extras.h mesa/src/compiler/glsl/ast_function.cpp mesa/include/GL/glcorearb.h mesa/src/compiler/glsl/glsl_parser_extras.cpp mesa/include/GL/glext.h mesa/src/compiler/glsl/glsl_parser_extras.h

Eleni Maria Stea Hacking on Mesa 3D 9 / 26 Developing the OpenGL GLSL/GLSL Compiler

In the previous slide we mentioned a ... GLSL compiler

What’s the purpose of using a compiler into a graphics API?

Eleni Maria Stea Hacking on Mesa 3D 10 / 26 Developing the OpenGL Rendering με το OpenGL : how we use the API

Remember that every OpenGL object is made by primitives!

Eleni Maria Stea Hacking on Mesa 3D 11 / 26 Developing the OpenGL The OpenGL Pipeline : Fully Programmable using the GLSL

http://chamilo2.grenet.fr/inp/courses/ENSIMAG4MMG3D/document/instructions/practical_01.html Eleni Maria Stea Hacking on Mesa 3D 12 / 26 Developing the OpenGL GLSL->GLSL IR->NIR->ASM

From OpenGL Shading Language to Assembly

▶ GLSL ▶ GLSL IR ▶ NIR : SSA (Single Static Assignment) + non SSA ▶ Native Code

Eleni Maria Stea Hacking on Mesa 3D 13 / 26 Developing the OpenGL GLSL->GLSL IR->NIR->ASM 1/4

Let us examine a very simple OpenGL program:

GLSL attribute vec4 vertex; void main() { gl_Position = vertex; }

GLSL IR

( (declare (location=0 shader_out ) vec4 gl_Position) (declare (location=17 shader_in ) vec4 vertex) ( function main (signature void (parameters ) ( (assign (xyzw) (var_ref gl_Position) (var_ref vertex) ) )) ) )

Eleni Maria Stea Hacking on Mesa 3D 14 / 26 Developing the OpenGL GLSL->GLSL IR->NIR->ASM 2/4

NIR (ssa form) decl_var shader_in INTERP_MODE_NONE vec4 vertex (VERT_ATTRIB_GENERIC0, 17, 0) decl_var shader_out INTERP_MODE_NONE vec4 gl_Position (VARYING_SLOT_POS, 0, 0) decl_function main returning void impl main { block block_0: /* preds: */ vec1 32 ssa_0 = load_const (0x00000000 /* 0.000000 */) vec4 32 ssa_1 = intrinsic load_input (ssa_0) () (0, 0) /* base=0 */ /* component=0 */ intrinsic store_output (ssa_1, ssa_0) () (0, 15, 0) /* base=0 */ /* wrmask=xyzw */ /* component=0 */ /* gl_Position */ /* succs: block_0 */ block block_0: }

Eleni Maria Stea Hacking on Mesa 3D 15 / 26 Developing the OpenGL GLSL->GLSL IR->NIR->ASM 3/4

NIR (final form) decl_var shader_in INTERP_MODE_NONE vec4 vertex (VERT_ATTRIB_GENERIC0, 17, 0) decl_var shader_out INTERP_MODE_NONE vec4 gl_Position (VARYING_SLOT_POS, 0, 0) decl_function main returning void impl main { block block_0: /* preds: */ vec1 32 ssa_0 = load_const (0x00000000 /* 0.000000 */) vec4 32 ssa_1 = intrinsic load_input (ssa_0) () (0, 0) /* base=0 */ /* component=0 */ intrinsic store_output (ssa_1, ssa_0) () (0, 15, 0) /* base=0 */ /* wrmask=xyzw */ /* component=0 */ /* gl_Position */ /* succs: block_0 */ block block_0: }

Eleni Maria Stea Hacking on Mesa 3D 16 / 26 Developing the OpenGL GLSL->GLSL IR->NIR->ASM 4/4

Native Code

START B0 (26 cycles) mov(8) g123<1>UD g1<8,8,1>UD { align1 WE_all 1Q compacted }; mov(8) g124<1>F g2<8,8,1>F { align1 1Q compacted }; mov(8) g125<1>F g3<8,8,1>F { align1 1Q compacted }; mov(8) g126<1>F g4<8,8,1>F { align1 1Q compacted }; mov(8) g127<1>F g5<8,8,1>F { align1 1Q compacted }; send(8) null<1>F g123<8,8,1>F urb 1 SIMD8 write mlen 5 rlen 0 { align1 1Q EOT }; nop ; END B0

Eleni Maria Stea Hacking on Mesa 3D 17 / 26 Debugging Tools GLSL->GLSL IR->NIR->ASM Debugging

We can test the intermediate representations using some Mesa environment variables for the drivers

$ export INTEL_DEBUG=vs $ export INTEL_DEBUG=fs $ export INTEL_DEBUG=vs,fs, ...

Like in the following example ;-)

Eleni Maria Stea Hacking on Mesa 3D 18 / 26 Debugging Tools OpenGL Extensions 4/4: Testing & Debugging

And since we mentioned Debugging...

There’s a process through which we can test our OpenGL extensions implementations.

▶ For every new core feature that is added to OpenGL, developers (usually from Khronos group) write a test that tests the feature. ▶ This test performs the same operation using and and compares the output of each. ▶ VK-GL-CTS: https://github.com/KhronosGroup/VK-GL-CTS ▶ Every manufacturer can run the tests on its own implementation of an OpenGL version to make sure that the features of that version work fine. ▶ When a test fails and the bug is not in the test, we know that probably our driver has a problem. ▶ We also check for regressions from version to version or for cases where a feature might break another one (a test that used to run without errors doesn’t do that anymore).

Eleni Maria Stea Hacking on Mesa 3D 19 / 26 Debugging Tools Other Debugging Tools

Some useful tools for when a problem is detected.

▶ GDB

▶ MESA environment variables: https://www.mesa3d.org/envvars.html • We can dump and replace information (asm, IR, GLSL code) • We can switch between GL versions • We can test different code paths We can test driver specific operations • only get a look at the website :) ▶ Some safe environments to run our experiments: chroot, jhbuild shell

Eleni Maria Stea Hacking on Mesa 3D 20 / 26 Debugging Tools Debuging mesa, debugging open source graphics apps, debugging anything!

We can debug any graphics app!!

Mesa provides debugging tools for: ▶ mesa itself ▶ other open source graphics applications ▶ even for !

Eleni Maria Stea Hacking on Mesa 3D 21 / 26 Debugging Tools Replacing the shaders’ code

Limbo Showcase:

No matter if we have a program’s source code, we can test but also modify its shaders while it’s running.

▶ $ export MESA_SHADER_DUMP_PATH=‘pwd‘/dump_sdr/ ▶ $ export MESA_SHADER_READ_PATH=‘pwd‘/read_sdr/

Eleni Maria Stea Hacking on Mesa 3D 22 / 26 Contributions How could someone get involved?

Coding Time!

▶ Mesa Repositories: https://cgit.freedesktop.org/mesa/

▶ Issue Tracking: https://www.mesa3d.org/bugs.html

▶ Debugging Tools: https://www.mesa3d.org/envvars.html

▶ Patches! (the process to submit them): https://www.mesa3d.org/submittingpatches.html

▶ Khronos CTS Repositories και Issue Tracker: https://github.com/KhronosGroup/VK-GL-CTS

Eleni Maria Stea Hacking on Mesa 3D 23 / 26 Contributions Some useful links

1 The mesa 3d graphics (Documentation/Tools): https://www.mesa3d.org/ 2 Khronos OpenGL registry: https://www.khronos.org/registry/OpenGL/index_gl.php 3 Intel microarchitectures: https://en.wikichip.org/wiki/intel/microarchitectures 4 Intel PRMs: https://01.org/linuxgraphics/documentation/hardware-specification-prms 5 Iago Toral’s introduction on mesa: https://blogs.igalia.com/itoral/category/graphics/ 6 Jasper St. Pierre’s blog post on Linux graphics stack: http://blog.mecheye.net/2012/06/the-linux-graphics-stack/ 7 Stéphane Marchesin’s book on Linux graphics drivers: https://people.freedesktop.org/~marcheu/linuxgraphicsdrivers.pdf 8 Samuel Iglesias’s posts on Piglit: https://blogs.igalia.com/siglesias/ 9 Alejandro Piñeiro’s posts on Mesa/Intel driver development we do at Igalia: https://blogs.igalia.com/apinheiro/ 10 Ben Widawsky’s post on GEN graphics and the URB: https://bwidawsk.net/blog/index.php/2015/09/gen-graphics-and-the-urb

Eleni Maria Stea Hacking on Mesa 3D 24 / 26 The End...

[email protected] http://eleni.mutantstargoat.com/hikiko

Eleni Maria Stea Hacking on Mesa 3D 25 / 26 The End...

Thank you very much!

Eleni Maria Stea Hacking on Mesa 3D 26 / 26