Linux display and video: one problem, three solutions, many puzzled developers Linux Plumbers Conference 2012 Laurent Pinchart [email protected] Problem Definition Problem Definition graphics / video Problem - Purpose format memory/deep pipeline device/CPU Problem - Source rotation scaling composing Problem - Processing X11 Wayland DirectFB Raw API Problem - Stack DRM FBDEV V4L2 Problem - API DRM == ?PC (mis)conceptions FBDEV is the only practical solution for embedded? platforms (mis)conceptions DRM requires a complex userspace ?stack (mis)conceptions Drivers should implement multiple interfaces? (mis)conceptions Frame Buffer Device Origins Video 4 Linux 2 Origins Direct Rendering Manager Origins Present DRM FB V4L2 Dynamic Allocation Yes No Yes Multiple Buffers Yes panning Yes Import dmabuf No userptr dmabuf Export mmap mmap mmap Memory Management DRM FB V4L2 KMS Yes Yes Yes RGB Formats 4CC 4CC 4CC Enumeration Yes No Yes Negotiation No No Yes Mode Setting DRM FB V4L2 Overlays Yes No Yes Rotation Yes No Yes Scaling Yes No Yes Cropping/Panning Yes Yes Yes Transformations DRM FB V4L2 2D No Console No 3D Yes No No Cursor Yes Yes No Composing No No No Acceleration drm fb v4l2 api core drivers Documentation The DRM core exports several interfaces to applications, “generally intended to be used through corresponding libdrm wrapper functions. In addition, drivers export device-specific interfaces for use by userspace drivers & device-aware applications through ioctls and sysfs files. External interfaces include: memory mapping, context management, DMA operations, AGP management, vblank control, fence management, memory management, and output management. Cover generic ioctls and sysfs layout here. We only need high- level info, since man pages should cover the rest. „ DRM API Documentation drm fb v4l2 api core drivers Documentation drm fb v4l2 api core drivers Code Size 4000 3500 3000 2500 drm fb 2000 v4l2 1500 1000 500 v4l2 fb 0 drm v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3 v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4 Cumulative Changes - API 45000 40000 35000 30000 drm 25000 fb 20000 v4l2 15000 10000 v4l2 5000 fb 0 drm v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3 v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4 Cumulative Changes - Core 1000000 900000 800000 700000 drm 600000 fb 500000 v4l2 400000 300000 200000 v4l2 100000 fb 0 drm v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3 v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4 Cumulative Changes - Drivers 1600 1400 1200 1000 drm 800 fb v4l2 600 drm+bug 400 200 0 2010-01 2010-05 2010-09 2011-01 2011-05 2011-09 2012-01 2012-05 2009-11 2010-03 2010-07 2010-11 2011-03 2011-07 2011-11 2012-03 2012-07 Mailing List Traffic Source: http://www.flickr.com/photos/buckaroobay/3721809183/ dmabuf Work In Progress dri2video https://wiki.linaro.org/OfficeofCTO/MemoryManagement? action=AttachFile&do=get&target=linux-video.pdf Targets: DRM Work In Progress panel & backlight http://lwn.net/Articles/512363/ Work In Progress Personal opinion Flame war possible Handle with care Disclaimer DRM needs to support dumb devices! better (mis)conceptions V4L2 still has use cases! (mis)conceptions FBDEV should be deprecated! (mis)conceptions • [email protected] • [email protected] • [email protected] • [email protected] Contact ? !.