Linux Display and Video: One Problem, Three Solutions, Many Puzzled Developers

Total Page：16

File Type：pdf, Size：1020Kb

Linux display and video: one problem, three solutions, many puzzled developers Linux Plumbers Conference 2012 Laurent Pinchart [email protected] Problem Definition Problem Definition graphics / video Problem - Purpose format memory/deep pipeline device/CPU Problem - Source rotation scaling composing Problem - Processing X11 Wayland DirectFB Raw API Problem - Stack DRM FBDEV V4L2 Problem - API DRM == ?PC (mis)conceptions FBDEV is the only practical solution for embedded? platforms (mis)conceptions DRM requires a complex userspace ?stack (mis)conceptions Drivers should implement multiple interfaces? (mis)conceptions Frame Buffer Device Origins Video 4 Linux 2 Origins Direct Rendering Manager Origins Present DRM FB V4L2 Dynamic Allocation Yes No Yes Multiple Buffers Yes panning Yes Import dmabuf No userptr dmabuf Export mmap mmap mmap Memory Management DRM FB V4L2 KMS Yes Yes Yes RGB Formats 4CC 4CC 4CC Enumeration Yes No Yes Negotiation No No Yes Mode Setting DRM FB V4L2 Overlays Yes No Yes Rotation Yes No Yes Scaling Yes No Yes Cropping/Panning Yes Yes Yes Transformations DRM FB V4L2 2D No Console No 3D Yes No No Cursor Yes Yes No Composing No No No Acceleration drm fb v4l2 api core drivers Documentation The DRM core exports several interfaces to applications, “generally intended to be used through corresponding libdrm wrapper functions. In addition, drivers export device-specific interfaces for use by userspace drivers & device-aware applications through ioctls and sysfs files. External interfaces include: memory mapping, context management, DMA operations, AGP management, vblank control, fence management, memory management, and output management. Cover generic ioctls and sysfs layout here. We only need high- level info, since man pages should cover the rest. „ DRM API Documentation drm fb v4l2 api core drivers Documentation drm fb v4l2 api core drivers Code Size 4000 3500 3000 2500 drm fb 2000 v4l2 1500 1000 500 v4l2 fb 0 drm v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3 v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4 Cumulative Changes - API 45000 40000 35000 30000 drm 25000 fb 20000 v4l2 15000 10000 v4l2 5000 fb 0 drm v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3 v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4 Cumulative Changes - Core 1000000 900000 800000 700000 drm 600000 fb 500000 v4l2 400000 300000 200000 v4l2 100000 fb 0 drm v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3 v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4 Cumulative Changes - Drivers 1600 1400 1200 1000 drm 800 fb v4l2 600 drm+bug 400 200 0 2010-01 2010-05 2010-09 2011-01 2011-05 2011-09 2012-01 2012-05 2009-11 2010-03 2010-07 2010-11 2011-03 2011-07 2011-11 2012-03 2012-07 Mailing List Traffic Source: http://www.flickr.com/photos/buckaroobay/3721809183/ dmabuf Work In Progress dri2video https://wiki.linaro.org/OfficeofCTO/MemoryManagement? action=AttachFile&do=get&target=linux-video.pdf Targets: DRM Work In Progress panel & backlight http://lwn.net/Articles/512363/ Work In Progress Personal opinion Flame war possible Handle with care Disclaimer DRM needs to support dumb devices! better (mis)conceptions V4L2 still has use cases! (mis)conceptions FBDEV should be deprecated! (mis)conceptions • [email protected] • [email protected] • [email protected] • [email protected] Contact ? !.

Copy Link

Recommended publications

OM-Cube Project
OM-Cube project V. Hiribarren, N. Marchand, N. Talfer [email protected] - [email protected] - [email protected] Abstract. The OM-Cube project is composed of several components like a minimal operating system, a multi- media player, a LCD display and an infra-red controller. They should be chosen to fit the hardware of an em- bedded system. Several other similar projects can provide information on the software that can be chosen. This paper aims to examine the different available tools to build the OM-Multimedia machine. The main purpose is to explore different ways to build an embedded system that fits the hardware and fulfills the project. 1 A Minimal Operating System The operating system is the core of the embedded system, and therefore should be chosen with care. Because of its popu- larity, a Linux based system seems the best choice, but other open systems exist and should be considered. After having elected a system, all unnecessary components may be removed to get a minimal operating system. 1.1 A Linux Operating System Using a Linux kernel has several advantages. As it’s a popular kernel, many drivers and documentation are available. Linux is an open source kernel; therefore it enables anyone to modify its sources and to recompile it. Using Linux in an embedded system requires adapting the kernel to the hardware and to the system needs. A simple method for building a Linux embed- ded system is to create a partition on a development host and to mount it on a temporary mount point. This partition is filled as one goes along and then, the final distribution is put on the target host [Fich02] [LFS].
[Show full text]
Institutionen För Datavetenskap Department of Computer and Information Science
Institutionen för datavetenskap Department of Computer and Information Science Final thesis Simulation of Set-top box Components on an X86 Architecture by Implementing a Hardware Abstraction Layer by Faruk Emre Sahin Muhammad Salman Khan LITH-IDA-EX—10/050--SE 2010-12-25 Linköpings universitet Linköpings universitet SE-581 83 Linköping, Sweden 581 83 Linköping Linköping University Department of Computer and Information Science Final Thesis Simulation of Set-top box Components on an X86 Architecture by Implementing a Hardware Abstraction Layer by Faruk Emre Sahin Muhammad Salman Khan LITH-IDA-EX—10/050—SE 2010-12-25 Supervisors: Fredrik Hallenberg, Tomas Taleus R&D at Motorola (Linköping) Examiner: Prof. Dr. Christoph Kessler Dept. Of Computer and Information Science at Linköpings universitet Abstract The KreaTV Application Development Kit (ADK) product of Motorola en- ables application developers to create high level applications and browser plugins for the IPSTB system. As a result, customers will reduce develop- ment time, cost and supplier dependency. The main goal of this thesis was to port this platform to a standard Linux PC to make it easy to trace the bugs and debug the code. This work has been done by implementing a hardware abstraction layer(HAL)for Linux Operating System. HAL encapsulates the hardware dependent code and HAL APIs provide an abstraction of underlying architecture to the oper- ating system and to application software. So, the embedded platform can be emulated on a standard Linux PC by implementing a HAL for it. We have successfully built the basic building blocks of HAL with some performance degradation.
[Show full text]
Openbricks Embedded Linux Framework - User Manual I
OpenBricks Embedded Linux Framework - User Manual i OpenBricks Embedded Linux Framework - User Manual OpenBricks Embedded Linux Framework - User Manual ii Contents 1 OpenBricks Introduction 1 1.1 What is it ?......................................................1 1.2 Who is it for ?.....................................................1 1.3 Which hardware is supported ?............................................1 1.4 What does the software offer ?............................................1 1.5 Who’s using it ?....................................................1 2 List of supported features 2 2.1 Key Features.....................................................2 2.2 Applicative Toolkits..................................................2 2.3 Graphic Extensions..................................................2 2.4 Video Extensions...................................................3 2.5 Audio Extensions...................................................3 2.6 Media Players.....................................................3 2.7 Key Audio/Video Proﬁles...............................................3 2.8 Networking Features.................................................3 2.9 Supported Filesystems................................................4 2.10 Toolchain Features..................................................4 3 OpenBricks Supported Platforms 5 3.1 Supported Hardware Architectures..........................................5 3.2 Available Platforms..................................................5 3.3 Certiﬁed Platforms..................................................7
[Show full text]
Audio Video Graphics Working Group Session
Audio Video Graphics Working Group Session San Jose meeting Pieter van de Meulen WG chair 26 January, 2005 CE Linux Forum Technical Conference 1 AVG WG – todays objective: Audio Video 2D 3D Renesas, Mitsubishi, Conexant DirectFB Philips, Samsung UH API P hilips Multi-FB …. OpenGL …. …. …. 26 January, 2005 CE Linux Forum Technical Conference 2 Recall: 25th/26th Presentations • DirectFB - Dennis Oliver Kropp; Convergence ● http://www.directfb.org/ • OpenGL ES, OpenVG and OpenMAX - Ed Plowman; ARM ● http://www.khronos.org/ • Graphics APIS for Linux - Matsubara, Hagiwara, Hisao Munakata; Renesas • Creating GTK+ based UI's for embedded devices - Markku Ursin, Movial ● http://www.gtk.org/ • Linux DVB - Michael Hunold; Convergence ● http://www.linuxtv.org/ • UHAPI (AV streaming) tutorial - John Vugts; Philips/Samsung ● http://www.uhapi.org/ 26 January, 2005 CE Linux Forum Technical Conference 3 Recall: Jan. 25th Demos • FB (Multi-framebuffer) ● Philips: PNX8550 running Linux; dual framebuffer and AV accel. • DirectFB ● Conexant: DVD processor running CELF Linux with DirectFB. ● Mitsubishi: ARIB plane model emulated on DirectFB window system ● Mitsubishi: MPEG4 Player (via GTK+) running on Renesas SH-4 ● Renesas: GTK+ and GUI without X11 • UHAPI (AV streaming): ● Philips open source demo on PC • DTV ● Toshiba America/Europe: DTV reference solution and Home Gateway. • 3D graphics ●Renesas: demo by SI-Electronics on SH-4 and Power VR ●Pioneer: OpenGL 26 January, 2005 CE Linux Forum Technical Conference 4 Linux APIs (2004/6 status) & CCEELLFF VV11..00
[Show full text]
Programmable Image-Based Light Capture for Previsualization
ii Abstract Previsualization is a class of techniques for creating approximate previews of a movie sequence in order to visualize a scene prior to shooting it on the set. Often these techniques are used to convey the artistic direction of the story in terms of cinematic elements, such as camera movement, angle, lighting, dialogue, and char- acter motion. Essentially, a movie director uses previsualization (previs) to convey movie visuals as he sees them in his ”minds-eye”. Traditional methods for previs include hand-drawn sketches, Storyboards, scaled models, and photographs, which are created by artists to convey how a scene or character might look or move. A recent trend has been to use 3D graphics applications such as video game engines to perform previs, which is called 3D previs. This type of previs is generally used prior to shooting a scene in order to choreograph camera or character movements. To visualize a scene while being recorded on-set, directors and cinematographers use a technique called On-set previs, which provides a real-time view with little to no processing. Other types of previs, such as Technical previs, emphasize accurately capturing scene properties but lack any interactive manipulation and are usually employed by visual eﬀects crews and not for cinematographers or directors. This dissertation’s focus is on creating a new method for interactive visualization that will automatically capture the on-set lighting and provide interactive manipulation of cinematic elements to facilitate the movie maker’s artistic expression, validate cine- matic choices, and provide guidance to production crews. Our method will overcome the drawbacks of the all previous previs methods by combining photorealistic ren- dering with accurately captured scene details, which is interactively displayed on a mobile capture and rendering platform.
[Show full text]
Directfb Internals Internals
DirectFB Internals --ThinThings You Need to Know to Write Your DirectFBgfxdriver TakanariHayama, HisaoMunakata and Denis Oliver Kropp 1 What is DirectFB?DirectFB? Thin Graphics Library – Light weight and small footprint (< 700KB Library for SH4) – Not server/client model like X11 Hardware Abstraction Layer for Hardware Graphics Acceleration – Anyygthing not supp ppyorted by Hardware still su ppypported by software, i.e. utilize hardware where possible Multi-process support And others, e.g. build-in Window Manager etc. 15-17 April, 2008 ELC 2008 2 The First Embedded Chip Support by the Mainline DirectFB – Renesas SH7722 15-17 April, 2008 ELC 2008 3 df_andi and SawMan running on SH7722 15-17 April, 2008 ELC 2008 4 DirectFB Software Architecture for Renesas SH4 Platform Applications Direct FB gfxdriver for system module – Renesas’s Platform devmem Existing Code Linux Kernel Kernel Module Custom Code Video Memory and Hardware Accelerator Hardware Hardware 15-17 April, 2008 ELC 2008 5 Important Terms in DirectFB Layers – Represents independent graphics buffers. Most of embedded devices have more than one layer. They get layered with appropriate alpha blending by hardware, and displayed. Surface – Reserved memory region to hold pixel data. Drawing and blitting operation in DirectFB is performed from/to surfaces. Memory of surfaces could be allocated from video memory or system memory depending on the given constraints. Primary Surface – Special s urface that represents frame bu ffer of particu lar lay er. If the primary surface is single buffered, any operation to this primary surface is directly visible to the screen. 15-17 April, 2008 ELC 2008 6 Concept of Layers Each represents layer On the screen they are layered.
[Show full text]
A Study on the Use of Opengl in Window Systems
Master Thesis Computer Science Thesis no: MCS-2004:03 March 2004 A study on the use of OpenGL in window systems Johan Persson Department of Software Engineering and Computer Science Blekinge Institute of Technology Box 520 SE - 372 25 Ronneby Sweden This thesis is submitted to the Department of Software Engineering and Computer Science at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 20 weeks of full time studies. Contact Information: Author: Johan Persson E-mail: [email protected] University advisor: Björn Törnqvist Department of Software Engineering and Computer Science Department of Software Engineering and Computer Science Internet : http://www.bth.se/ipd Blekinge Institute of Technology Phone : +46 457 38 50 00 Box 520 Fax : + 46 457 271 25 SE - 372 25 Ronneby Sweden Abstract OpenGL is getting used more in window system as a way of improving performance and enabling new functionality. Examples of two systems using different approaches of how OpenGL is being used are Quartz Extreme and Fresco. Quartz Extreme uses window composition which assures fast redisplay and Fresco on the other hand uses a structured graphics approach which utilises OpenGL on a much lower level compared to Quartz Extreme which operates at a window level. Fresco’s way brings great flexibility and an ability to mix 2D and 3D-objects on the desktop. But each of the approaches has its problems; Quartz extreme requires a large amount of memory for buffering of the window contents and Fresco has performance problems when redisplaying complex structures.
[Show full text]
La Pile Graphique Linux
La pile graphique Linux Jérôme Pouiller <[email protected]> Programme Rappels sur les OS Historique du protocole X (la préhistoire) Accélération 2D Accélération des vidéos Accélération 3D des clients Drivers graphiques 3D Accélération du bureau toolkits graphiques Sysmic - J. Pouiller La pile graphique Linux 2 / 29 Rappel sur la mémoire Les CPU modernes intègrent un composant appelé MMU (Memory Management Unit): Unité de translation d’adresses mémoire On parle d’adresses physiques et virtuelles Lorsque le MMU est actif (cas nominal), toutes les adresses du code assembleur sont des adresses virtuelles Il est possible de conﬁgurer le MMU avec une instruction spéciale et en lui donnant un pointeur sur un tableau (dans la pratique, il s’agit plutôt d’un arbre) associant les adresses physiques et les adresses virtuelles Sysmic - J. Pouiller La pile graphique Linux 3 / 29 Drivers en espace utilisateur mmap permet d’allouer des plage d’adresses virtuelles Elles sont associées à : Mémoire anonyme (= allocation de mémoire) Fichiers (en intégralité ou en partie) /dev/mem représente la mémoire physique Utiliser mmap avec /dev/mem revient à accéder directement aux adresses physiques Avec iopl et ioperm, il est possible à un programme utilisateur d’accéder au ioport (inb, outb, ...) XFree86 est basé sur ce modèle Sysmic - J. Pouiller La pile graphique Linux 4 / 29 Fonctionnement de XFree86 Permet d’être portable entre les différents OS X gère aussi l’accès partagé aux périphériques d’entrées (souris, clavier, ’ecran tactile) C’est normalement le rôle de l’OS de gérer les accès aux périphériques Accéder directement aux périphériques peut rentrer en conﬂit avec la gestion de l’OS (plantage) Au fur et a mesure du temps, on a porté des morceaux des pilotes de X dans le noyau : Au lieu de travailler sur /dev/mem, on travaille sur /dev/nvidia.
[Show full text]
I.MX Graphics Users Guide Android
NXP Semiconductors Document Number: IMXGRAPHICUG Rev. 0, 02/2018 i.MX Graphics User’s Guide Contents Chapter 1 Introduction ............................................................................................................................................. 6 Chapter 2 i.MX G2D API ............................................................................................................................................ 6 2.1 Overview ...................................................................................................................................................... 6 2.2 Enumerations and structures ....................................................................................................................... 6 2.3 G2D function descriptions .......................................................................................................................... 10 2.4 Support of new operating system in G2D .................................................................................................. 16 2.5 Sample code for G2D API usage ................................................................................................................. 16 2.6 Feature list on multiple platforms.............................................................................................................. 19 Chapter 3 i.MX EGL and OGL Extension Support .................................................................................................... 20 3.1 Introduction ..............................................................................................................................................
[Show full text]
The History of Embedded Linux & Best Practices for Getting Started
The History of Embedded Linux & Best Practices for Getting Started Henry Kingman July 2013 A Linux Foundation Training Publication www.training.linuxfoundation.org DISCLAIMER The Linux Foundation frequently works with subject matter experts on Linux educational pieces. In this publication, famed Linux Devices’ editor Henry Kingman has created a fun history and background on embedded Linux; we hope you enjoy Henry’s unique perspective on this fast-moving industry. The opinions found in this publication are Henry Kingman’s alone and don’t necessarily reflect those at the Linux Foundation. Overview This paper offers historical perspective and advice for those considering or planning to embed Linux. Before we plunge in, let’s define some terms: • By “Linux,” we mean any stack that includes a Linux kernel. Besides the kernel itself, stacks typically include a great deal of other software – typically the GNU C libraries, some graphics stack (like xorg), and the vendor’s particular software “value add.” However, a Linux device may include little more than a bootloader, bootstrap loader, and initramfs (i.e., a Linux kernel and a module or two running on a ramdisk). • By “embedded device,” we mean any device with a microprocessor, exclusive of PCs and servers. This describes the vast majority of computerized systems, since by most analyst estimates, PCs and servers consume only 1-5 percent of all microprocessor chips. The rest go into that vast, diverse, hard-to-generalize-about category known broadly as the “embedded” or “device” market. Many such systems, perhaps two thirds, have 8- and 16-bit “microcontrollers” that do not support “rich” 32-bit OSes like Linux.
[Show full text]
Systèmes Et Applications Embarquées
Free Software for Embedded Systems Mail: [email protected] Web: http://stephane.lavirotte.com/ University of Nice - Sophia Antipolis Introduction We must start with something… 3/31/2010 Free Electrons - Stéphane Lavirotte 2 Linux for Embedded Systems 9 Linux for embedded systems – Is used more and more – In use in many devices Mobile Phones PDA (Personal Digital Equipment) Network Equipments … Have a look to: – http://www.linuxfordevices.com/c/a/Linux-For-Devices-Articles/The-Linux-Devices-Showcase/ – Used in limited resources systems uClinux, uClibc, dietlib, … 9 Easy to include free software inside that products – sqlite, samba, thttpd, … 3/31/2010 Free Electrons - Stéphane Lavirotte 3 Development Phases 9 Configuration – First, choose which features to include in the system Kernel configuration Choose the tools and applications to include 9 Build – Need to recompile the entire system for the target: Kernel Libraries Applications 9 Deployment – Installation of files in a filesystem – Transfer on the target and start the new system 3/31/2010 Free Electrons - Stéphane Lavirotte 4 System Components 9 Component – Boot Loader Boot Loader – Kernel Kernel – Kernel Modules – File Systems Kernel Modules – Libraries Libraries – Applications Applications File System 3/31/2010 Free Electrons - Stéphane Lavirotte 5 About Free Software 9 Linux is and Open Source Free Software 9 Free Software provides 4 freedom to the user: – The freedom to use the program as it sees fit – The freedom to study how the program works and adapt it to its own needs (Get the source code is a precondition). – The freedom to redistribute copies to help others – The freedom to improve the program, and distribute the improvements to the public so that the community can benefit from advanced (Get the source code is a precondition).
[Show full text]
Multimedia in Embedded Linux Systems
Multimedia in embedded Linux systems Multimedia in embedded Linux systems Free Electrons 1 Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//free-electrons.com Rights to copy © Copyright 2004-2009, Free Electrons [email protected] Document sources, updates and translations: http://free-electrons.com/docs/multimedia Corrections, suggestions, contributions and translations are welcome! Attribution ± ShareAlike 3.0 Latest update: Sep 15, 2009 You are free to copy, distribute, display, and perform the work to make derivative works to make commercial use of the work Under the following conditions Attribution. You must give the original author credit. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. License text: http://creativecommons.org/licenses/by-sa/3.0/legalcode 2 Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//free-electrons.com Scope of this training Multimedia in embedded Linux systems This training targets the development of multimedia-capable embedded Linux systems. Though it can be useful to playing or creating multimedia content on GNU/Linux desktops, it is not meant to cover everything about this topic. Audio in embedded Linux systems is already covered in a dedicated training: http://free-electrons.com/training/audio Linux 2.6 This training only targets new systems based on the Linux 2.6 kernel.
[Show full text]