!1 67 Tizen v2.3 Linux Kernel Sungkyunkwan University Embedded Software Lab. @ SKKU !2 Tizen 内核附加特性 49 • 存储管理 – CMA (连续的内存分配器 ), IOMMU, dma_buf, DRM (直 接渲染管理器 ) • ARM Linux没有 DMA_ZONE伙伴 • 有些设备需要物理上的连续的内存 (CMA), 其他设备有 IOMMU从⽽ 能够使⽤不连续的内存 • 对于 D/D⽤户可以共享 DMA存储缓存 – 电源管理 (CPUfreq, Devfreq, PM-QoS, Charger- manager) – KDBUS – V4L2 (Video for Linux 2) – EXTCON Embedded Software Lab. @ SKKU !3 以前参考内核 49 • Linux 3.0.15 – Obsolete LTS (Current: 3.4 & 3.10) • ⽀持 RD-PQ (Tizen 2) & RD-210 (tizen 1 & 2) – RD-PQ: Exynos4412 – RD-210: Exynos 4210 (Linux 2.6.36 for Tizen 1) • 不好的参照 – 太多的补丁功能 – 太陈旧！ 不⽀持 LTS/LTSI – 许多内核⿊客和肮脏的补丁 – git历史删除 • 从⽣产内核中 Fork • 很难阅读 Embedded Software Lab. @ SKKU Linux Kernel. LTS? LTSI? !4 67 • Long-Term Stable (LTS) – Maintained by Greg K.H. – Bugfixes for 2 years or longer. – Up to 2 LTS kernels at the same time. – Recent: 3.10.39 (2014/5/6) • Long-Term Stable Initiative (LTSI) – Maintained by Greg K.H. and some manufacturers – Forked LTS for Industry. (LTS + Industry Patchset) – Maintain a common Linux base for use in a variety of consumer electronics products – Longer support period. – Recent: 3.10.31-LTSI (2014/2/24) Embedded Software Lab. @ SKKU !5 Tizen 参考内核的状态 49 • 两个参考内核 : ARM/Intel • ARM (armv7, aarch64) – Linux 3.10.y • 3.10.33 @ 2014/05 • 完整的 git历史 – 即将推出 Armv6 support (Raspberry Pi) – 测试 &验证阶段 (集成测试和⽤户空间 ) • Intel (x86, x86_64) – Linux 3.14.1 – Recent ATOM Soc support merged Embedded Software Lab. @ SKKU !6 Tizen 内核概述 49 Embedded Software Lab. @ SKKU !7 Tizen 内核概述 (续 ) 49 Embedded Software Lab. @ SKKU Static Memory vs. CMA memory Multimedia !8 example 67 dma_alloc_coherent() Embedded Software Lab. @ SKKU ION !9 67 • generalized memory manager that Google introduced in the Android 4.0 ICS • Provides way for userland to allocate buffers from various “pools of memory” (aka: heaps) • CMA also allows for large physically contiguous memory allocations by migrating memory to make room for the large allocation – CMA is kernel-internal only for now, and doesn't have a interface to allow userland to allocate buffers or specify constraint options – Migrating pages to make room can cause non-deterministic delays. Embedded Software Lab. @ SKKU ION !10 67 Embedded Software Lab. @ SKKU !11 67 Tizen v2.3 内存管理 Embedded Software Lab. @ SKKU !12 Tizen 存储管理 49 • 连同图形和多媒体设备 – 图形⾦额多媒体设备 = 有巨⼤缓存的 DMA设备 Embedded Software Lab. @ SKKU !13 DRM是什么 49 • DRM不是数字版权管理 • 直接渲染管理器 – ⽀持直接渲染管理器结构的内核级别图形设备 – 直接渲染 • 应⽤直接与图形设备驱动交互 – 显⽰模式设置 – 图形存储管理 Embedded Software Lab. @ SKKU !14 为什么使⽤ DRM 49 • 可以提供类似于 PC环境的展⽰体验 • 直到现在为什么不使⽤ DRM？ – DRM是为计算机设计的 – 嵌⼊式系统 • 性能低 • 内有专⽤图形内存 • 没有⼀个显卡却⼜分开硬件设备 – Linux帧缓冲驱动程序 Embedded Software Lab. @ SKKU !15 为什么使⽤ DRM (续 ) 49 • 改变了嵌⼊式系统 – 强⼤的嵌⼊式 SoCs – 需求 • 显⽰热插拔与克隆拓展模式 • 统⼀内存管理 • 直接渲染 • 不同的设备控制与公共接⼝ Embedded Software Lab. @ SKKU X11 infrastructure !16 67 DIX (Device-Independent X) DDX (Device-Dependent X), Embedded Software Lab. @ SKKU Early implementation of the Linux graphics stack !17 67 UMS (User-space Mode-Setting) Embedded Software Lab. @ SKKU Early implementation of the Linux graphics stack !18 67 • XFree86 server: the first Linux 2D graphics hardware acceleration – Super-user privileges – Access the card directly from user space, no kernel support – Simple and portable • Utah-GLX: the first hardware-independent 3D accelerated design – User space 3D driver, directly accesses the graphics hardware from user space – 3D hardware was clearly separated from 2D (3Dfx), completely separate driver. • Framebuffer drivers – Another component that could simultaneously access the graphics hardware directly. – To avoid potential conflicts between the framebuffer and XFree86 drivers, – On VT(virtual terminal) switches, the kernel would emit a signal to the X server telling it to save the graphics hardware state. – More fragile and bug-prone drivers. • Drawbacks – Security: unprivileged user space applications be allowed to access the graphics hardware for 3D. – Performance: all GL acceleration had to be indirect through the X protocol Embedded Software Lab. @ SKKU DRI model !19 67 Embedded Software Lab. @ SKKU DRI/DRM infrastructure !20 67 • To address the reliability and security concerns with the Utah-GLX model • Additional kernel component whose duty is to check the correctness of the 3D command stream, security-wise. • Instead of accessing the card directly, the unprivileged OpenGL application would submit command buffers to the kernel, which would check them for security and then pass them to the hardware for execution. • Trusting user space is no longer required. • Kernel Mode-Setting (KMS) – Merge the kernel framebuffer functionality into the DRM module – X.Org access the graphics card through the DRM module and run unprivileged. – DRM module is now responsible for providing mode-setting services both as a framebuffer driver and to X.Org. Mode-setting: configuring video mode to get a picture on the screen. includes choosing the video resolution and refresh rates. Embedded Software Lab. @ SKKU Framebuffer Drivers !21 67 • Kernel graphics driver exposing its interface through / dev/fb* • Despite their simplicity, framebuffer drivers are still a relevant option for basic 2D display. • Especially for embedded systems – memory footprint is essential – do not require advanced graphics acceleration. • Functionalities – Mode-setting – Optional 2D acceleration • http://www.linux-fbdev.org/HOWTO/index.html Embedded Software Lab. @ SKKU Direct Rendering Manager !22 67 • Put critical initialization of the card in the kernel, for example uploading firmwares or setting up DMA areas. • Share the rendering hardware between multiple user space components, and arbitrate access. • Enforce security by preventing applications from performing DMA to arbitrary memory regions, and more generally from programming the card in any way that could result in a security hole. • Manage the memory of the card, by providing video memory allocation functionality to user space. • More recently, the DRM was improved to achieve mode-setting. – Simplifies the previous situation where both the Libdrm interfaces between user space and the DRM DRM and the framebuffer driver were fighting module, unprivileged user space component to access the same GPU. – Instead, the framebuffer driver is removed and framebuffer support is implemented in the DRM. Embedded Software Lab. @ SKKU DRM of Tizen v2.3 kernel !23 67 • Currently, DRM support is only for Exynos SoCs. (ARM based) – Need Exynos specific DRM driver • Exynos DRM driver – To support graphics hardware of Exynos SoCs – First ARM SoC graphics driver to use the DRM – Merged into the mainline linux 3.2 kernel – linux/drivers/gpu/drm/exynos Embedded Software Lab. @ SKKU !24 图形 (DRM / GEM) 49 • 图形 : – DRM (直接渲染管理 ) / GEM (图形处理管理 ) GEM •由 Intel开发的框架 • 为了管理图形存储 •缓存管理框架 • 分配和共享 Embedded Software Lab. @ SKKU !25 DRM / GEM 分配 49 GEM 分配步骤 @ Tizen (Generic) •DRM_IOCTL_MODE_CREATE_DUMB – 创建 GEM全局 & user GEM handle(per process) • dumb_create() of struct drm_driver – 没有物理内存分配 •DRM_IOCTL_MODE_MAP_DUMB – 创建虚假 mmap offset of a gem object 并将这个对象传递给⽤户 • A hash key of the gem object. • dumb_map_offset() of struct drm_driver •MMAP – 请求 mmap基于哈希键作为偏移量 – 创建⽤户地址空间 – 设置缓存属性 – 还没有映射到物理内存 Embedded Software Lab. @ SKKU !26 DRM / GEM 分配 (续 ) 49 GEM 分配步骤 @ Tizen (Generic) • 按需分⻚ – 实现和注册⼀个错误处理程序 • With a page fault, allocate a page and map the page. – vma->vm_ops->fault = xxx_drm_gem_fault •使⽤ ! •DRM_IOCTL_MODE_DESTROY_DUMB – 移除 GEM handle & object – 释放内存 – 实现 dumb_destroy() of struct drm_driver Embedded Software Lab. @ SKKU !27 DRM / GEM 分配 (续 ) 49 GEM 分配步骤 @ Tizen (Exynos Only) •DRM_IOCTL_EXYNOS_GEM_CREATE – 只使⽤⽤户需要的⼤⼩和缓存类型 – 创建 gem全局对象和 user gem handle(per process) – 分配物理内存 •DRM_IOCTL_EXYNOS_GEM_MMAP – 创建⽤户地址空间 – 将⽤户地址空间映射到物理内存 •LIBDRM of Exynos uses these APIs, not the generic. Embedded Software Lab. @ SKKU !28 DRM / GEM 共享 49 GEM 共享 @ Tizen •DRM_IOCTL_GEM_FLINK – “I will share this GEM to others.” – Create GEM object name for the given GEM handle • 共享全局键值 •DRM_IOCTL_GEM_OPEN – “I want to use the shared GEM.” – 基于已有 GEM对象名称，创建 Create GEM handle •DRM_IOCTL_GEM_CLOSE Embedded Software Lab. @ SKKU !29 Tizen Kernel MM –Multimedia (V4L2 / VB2) 49 Embedded Software Lab. @ SKKU !30 V4L2 / VB2 49 • 对于多媒体设备， Tizen推荐在内核中使⽤ – Video input (codec & camera) & Radio – 然⽽，尽管内核具有 ..: • Gstreamer/OpenMAX plugins • A method to share with other F/W via DMABUF of UMM • 如果使⽤ V4L2/VB2, 事情会变得更简单 Embedded Software Lab. @ SKKU !31 Tizen Kernel MM –OpenGL / G3D-GPU 49 Embedded Software Lab. @ SKKU !32 OpenGL / G3D-GPU 49 • ⼤部分 ARM SoC GPUs (MALI, SGX, …)使⽤ 它们⾃⼰的内存 – 例如 , Exynos4412/4210 Tizen Reference Kernel 有 Mali w/ UMM. • Mali DDK modified to be compatible with UMM-DMABUF. • 如果 GPU驱动使⽤ DRM, 将会更好 . – (and make them GPL) Embedded Software Lab. @ SKKU !33 图形化和显⽰器使⽤场景 49 Embedded Software Lab. @ SKKU !34 图形化和显⽰器使⽤场景 49 Embedded Software Lab. @ SKKU !35 图形化和显⽰器使⽤场景 49 Embedded Software Lab. @ SKKU !36 图形化和显⽰器使⽤场景 49 Embedded Software Lab. @ SKKU !37 图形化和显⽰器使⽤场景 49 Embedded Software Lab. @ SKKU !38 Tizen 内核缓冲共享 49 Embedded Software Lab. @ SKKU !39 Tizen 内核缓冲共享 49 • Tizen平台和硬件的需求 – 不同的内存管理器 : GEM, VB2, GPU-adhoc,… – 共享缓冲 • w/o memcpy • From and to users • 不要直接暴露给⽤户 (e.g., 物理地址 ) ➔UMM DMABUF! Embedded Software Lab. @ SKKU !40 统⼀的内存管理器 (UMM) 49 • 由 Jesse Barker引⼊ , 2011 • 包含 – DMABUF (共享缓冲 ) • Export – GEM/VB2/… object ! DMABUF • Import – DMABUF ! GEM/VB2/… object • ⽤户空间将 DMABUF当作⽂件描述器 – DMA mapping API for allocation – CMA (Contiguous Memory Allocator) – IOMMU (MMU for I/O devs) Embedded Software Lab. @ SKKU !41 DMA缓冲共享 : Camera ! Display 49 Embedded Software Lab. @ SKKU !42 DMA 缓冲共享⽰例 49 • Camera App 1)请求 V4L2 相机缓存 (U) 2)分配 CMA 缓存 (K) 3)向 V4L2缓存请求⼀个 camera 帧 (U) 4存储 camera帧 & 通知⽤户 (K) 5)请求 DMABUF 为 V4L2 camera 缓冲导出 (U） 6)dma_buf_exporter() (K) • Create DMABUF from V4L2 buffer 7)dma_buf_fd() (K) • Provide FD of the DMABUF to user Embedded Software Lab. @ SKKU !43 DMA缓冲共享⽰例 49 • Camera App 1)请求 V4L2 camera 缓冲 (U) 2)分配 CMA 缓冲 (K) 3)向 V4L2缓存请求⼀个 camera 帧 (U) 4)存储 camera帧 & 通知⽤户 (K) 5)请求 DMABUF 为 V4L2 camera 缓冲导出 (U） 6)dma_buf_exporter() (K) • Create DMABUF from V4L2 buffer 7)dma_buf_fd() (K) • Provide FD of the DMABUF to user Embedded Software Lab.