Creating Digital Products Part 2 : Software Cambridge Game Creators London – 2015-05-21 Laurent Pinchart [email protected] Problem Definition Problem Definition Problem Definition Linux System Boot Loader Up & Running Kernel Initialization 0 1 4 15 time Graphics Use Cases Fixed Logo Boot Loader Fixed Logo Linux Kernel Animation, Feedback System Initialization Full Graphics Stack Up & Running Traditional Linux Graphics Architecture Qt GTK+ EFL Apps Xlib Xorg X server DDX Mesa 3D libdrm FBDEV DRM/KMS Kernel Hardware X11 Graphics Stack ● DDX – Device Dependent X driver. Handles initialization, manages the display, performs 2D accelerated rendering. ● DRM – Direct Rendering Manager. Subsystem of the Linux kernel responsible for interfacing with GPUs. ● EFL, GTK+, Qt – Open-source graphical toolkits. ● EGL – Interface between rendering APIs and the underlying native platform windowing system. ● FBDEV – Frame Buffer Device. Subsystem of the Linux kernel responsible for accessing the display framebuffer. ● GLES – GL Embedded System. Embedded profile of the OpenGL API. ● KMS – Kernel Mode Setting. Subsystem of the Linux kernel responsible for interfacing with display controllers. Handles mode setting and hardware composition. ● Mesa (3D) – Open-source implementation of the EGL, GL, GLES, … APIs. ● Xlib – Reference implementation of the client side of the X11 protocol. ● Xorg – Reference implementation of the server side of the system. Acronyms X11 Client . X11 Client Compositor X Server FBDEV evdev DRM/KMS X11 Composition X11 Client . X11 Client Compositor X Server 1 FBDEV evdev DRM/KMS X11 Composition X11 Client . X11 Client 2 3 Compositor X Server 1 FBDEV evdev DRM/KMS X11 Composition X11 Client . X11 Client 2 3 4 Compositor X Server 1 FBDEV evdev DRM/KMS X11 Composition X11 Client . X11 Client 2 3 4 Compositor 5 X Server 1 FBDEV evdev DRM/KMS X11 Composition X11 Client . X11 Client 2 3 4 Compositor 5 X Server 1 6 FBDEV evdev DRM/KMS X11 Composition Linux Kernel Display APIs FBDEV vs. DRM APIs Frame Buffer Display (memory) Device Device Model – FBDEV Planes CRTC Encoder Connector (memory) Device Model – DRM/KMS Source: http://xkcd.com/619/ Features DRM FB Dynamic Allocation Yes No Multiple Buffers Yes panning Import dmabuf No dmabuf Export mmap mmap Memory Management DRM FB RGB Formats 4CC 4CC Enumeration Planes No Negotiation No No Atomicity Yes No Mode Setting DRM FB Overlays Yes No Rotation Yes No Scaling Yes No Cropping/Panning Yes Yes Transformations DRM, Linux 2.2.18 2000 Activity – DRM/KMS GEM 2008 KMS, TTM 2009 Dumb Buffers 2011 Planes, DMABUF 2012 Atomic mode setting 2013-... Sync & fences FBDEV, Linux 1.3.94 1996 Blanking 2000 Activity – FBDEV 4CC Formats 2012 4CC Formats 2012 Activity – FBDEV Source: http://valdodge.com/2010/07/ 3000 2500 2000 1500 drm fb drm+bug 1000 500 0 Mailing List Traffic 2000 1800 1600 1400 1200 drm 1000 fb 800 600 400 200 fb 0 drm Cumulative Changes - API 70000 60000 50000 40000 drm fb 30000 20000 10000 fb 0 drm Cumulative Changes - Core 1800000 1600000 1400000 1200000 1000000 drm 800000 fb 600000 400000 200000 fb 0 drm Cumulative Changes - Drivers Source: http://xkcd.com/927/ Use Cases Use Cases - FBDEV (that's it...) Use Cases - FBDEV Everything Use Cases – DRM/KMS Beyond X11 X11 Client . X11 Client 2 3 4 Compositor 5 X Server 1 6 DRM/KMS evdev FBDEV X11 Composition by Shmuel Csaba Otto Traian; GNU FDL 1.2+ and CC-BY-SA 3.0+; created 2013-08-28; updated 2014-02-27 GNOME Video Clutter 1.17.4 W2EGL EGL glue libwayland-client code Inkscape GTK+ 3.12 W2EGL EGL glue libwayland-client code KDE Plasma Qt 5.3 W2EGL EGL glue libwayland-client code EGL libwayland-server Wayland Compositor libwayland-EGL libEGL-mesa-drivers Weston, Mutter, KWin, Clayland, libEGL-mesa EGL libGBM Enlightenment, ... D 3 OpenGL|ES libGLES-mesa libOpenVG-mesa libinput a OpenVG s e M Graphics device drivers libGL-mesa-DRI ① ④ ioctl libDRM evdev KMS DRM Kernel CPU & main memory USB, PS/2, ... GPU & graphic memory framebuffer Wayland Graphics Stack Wayland Wayland Client . Client 2 3 Wayland Compositor 1 4 evdev DRM/KMS Wayland Composition Single GUI Application Toolkit (Qt, SDL, ...) DRM/KMS evdev FBDEV Custom Application The DRM and KMS APIs DRM ? KMS APIs ● Memory Management ● Command Stream ● Vertical Blanking ● Version, Authentication, Master, ... DRM ● Device Model ● Frame Buffer ● Modes ● Page Flip ● Planes ● Cursor, Gamma, ... KMS drm kms armada exynos i810 i915 mga msm nouveau omap qxl r128 radeon savage sis tegra via DRM/KMS API Planes CRTC Encoder Connector (memory) Connector Encoder Connector Device Model Planes CRTC Encoder Connector (memory) Encoder Connector Memory SoC Off-Chip Device Model - SoC Frame Buffer KMS – Scanout Root Plane Composition Overlay Plane(s) KMS – Composition Frame Buffer ● width ● height GEM ● format Object(s) ● pitches CRTC ● offsets Memory Properties KMS – Frame Buffer Frame Buffer GEM Object ● width ● height CRTC ● bpp Memory ● pitch ● size Properties DRM/KMS – GEM Object Process A Process B 3 Send FD SCM_RIGHTS Local Global Global Local 1 2 4 Handle FD FD Handle GEM Object DRM – Handles sync back porch active area Active Area front porch KMS – Modes (1/2) hdisplay hsync_start hsync_end htotal sync back porch active area Active Area front porch KMS – Modes (2/2) fb x mode m o d e . Active v y d i Area s p crtc l a y mode.hdisplay *connectors struct drm_mode_set { num_connectors struct drm_framebuffer *fb; struct drm_crtc *crtc; struct drm_display_mode *mode; uint32_t x; uint32_t y; struct drm_connector **connectors; size_t num_connectors; }; KMS – Mode Setting KMS Programming Example Code Ahead Error handling omitted for readability Disclaimer #include <stdio.h> #include <stdlib.h> #include <stdint.h> #include <string.h> int main(int argc, char **argv) { return 0; } Skeleton /* * Open * * #include <xf86drm.h> * * int drmOpen(const char *name, const char *busid); */ int fd; fd = drmOpen("imx-drm", NULL); Open /* * Get resources * * #include <xf86drmMode.h> * * drmModeResPtr drmModeGetResources(int fd); * void drmModeFreeResources(drmModeResPtr ptr); */ drmModeResPtr resources; uint32_t crtc_id; uint32_t connector_id; resources = drmModeGetResources(fd); crtc_id = resources->crtcs[0]; connector_id = resources->connectors[0]; Get Resources /* * Get modes * * #include <xf86drmMode.h> * * drmModeConnectorPtr drmModeGetConnector(int fd, * uint32_t connectorId); * void drmModeFreeConnector(drmModeConnectorPtr ptr); */ drmModeConnectorPtr connector; drmModeModeInfo mode; uint32_t width; uint32_t height; connector = drmModeGetConnector(fd, connector_id); mode = connector->modes[0]; width = mode.hdisplay; height = mode.vdisplay; Get Modes /* * Create buffer * * #include <drm.h> * #include <drm_mode.h> * * #define DRM_IOCTL_MODE_CREATE_DUMB ... * * int drmIoctl(int fd, unsigned long request, void *arg); */ struct drm_mode_create_dumb bo_create; memset(&bo_create, 0, sizeof(bo_create)); bo_create.bpp = 32; bo_create.width = width; bo_create.height = height; drmIoctl(fd, DRM_IOCTL_MODE_CREATE_DUMB, &bo_create); Create Buffer /* * Fill buffer * * #include <drm.h> * #include <drm_mode.h> * #include <xf86drm.h> * * #define DRM_IOCTL_MODE_MAP_DUMB ... * * int drmIoctl(int fd, unsigned long request, void *arg); */ struct drm_mode_map_dumb bo_map; memset(&bo_map, 0, sizeof(bo_map)); bo_map.handle = bo_create.handle; drmIoctl(fd, DRM_IOCTL_MODE_MAP_DUMB, &bo_map); Fill Buffer (1/2) /* * Fill buffer * * #include <sys/mman.h> * * void *mmap(void *addr, size_t length, int prot, * int flags, int fd, off_t offset) * int munmap(void *addr, size_t length); */ void *mem; mem = mmap(0, bo_create.size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, bo_map.offset); fill_pattern(DRM_FORMAT_ARGB8888, mem, width, height, bo_create.pitch); munmap(mem, bo_create.size); Fill Buffer (2/2) /* * Create frame buffer * * #include <drm_fourcc.h> * #include <xf86drmMode.h> * * int drmModeAddFB2(int fd, uint32_t width, * uint32_t height, uint32_t pixel_format, * uint32_t bo_handles[4], uint32_t pitches[4], * uint32_t offsets[4], uint32_t *buf_id, * uint32_t flags); */ uint32_t handles[4] = { bo_create.handle, }; uint32_t pitches[4] = { bo_create.pitch, }; uint32_t offsets[4] = { 0, }; uint32_t fb_id; drmModeAddFB2(fd, width, height, DRM_FORMAT_ARGB8888, handles, pitches, offsets, &fb_id, 0); Create Frame Buffer /* * Set the mode * * #include <xf86drmMode.h> * * int drmModeSetCrtc(int fd, uint32_t crtcId, * uint32_t bufferId, uint32_t x, uint32_t y, * uint32_t *connectors, int count, * drmModeModeInfoPtr mode); */ drmModeSetCrtc(fd, crtc_id, fb_id, 0, 0, &connector_id, 1, &mode); Set the Mode /* * Get plane resources * * #include <xf86drmMode.h> * * drmModePlaneResPtr drmModeGetPlaneResources(int fd); * void drmModeFreePlaneResources(drmModePlaneResPtr ptr); */ drmModePlaneResPtr planes; uint32_t plane_id; planes = drmModeGetPlaneResources(fd); plane_id = planes->planes[0]; Get Plane Resources /* * Set the plane * * #include <xf86drmMode.h> * * int drmModeSetPlane(int fd, * uint32_t plane_id, uint32_t crtc_id, * uint32_t fb_id, uint32_t flags, * uint32_t crtc_x, uint32_t crtc_y, * uint32_t crtc_w, uint32_t crtc_h, * uint32_t src_x, uint32_t src_y, * uint32_t src_w, uint32_t src_h); */ drmModeSetPlane(fd, plane_id, crtc_id, fb_id, 0, width / 4, height / 4, width / 2, height / 2, 0, 0, (width / 2) << 16, (height / 2) << 16); Set the Plane /* * List plane properties * * #include <drm_mode.h> * #include <xf86drmMode.h> * * drmModeObjectPropertiesPtr drmModeObjectGetProperties( * int fd, uint32_t object_id, uint32_t object_type);