Memory Management in

SW Platform Team, SW R&D Center Contents

• Tizen Kernel Overview • Memory Management in Tizen Kernel • Memory Size Optimization

2 tizen.org Tizen Kernel Overview

3 tizen.org Tizen Kernel Overview

Core

Multimedia Telephony Sensor FW System FW . . . FW FW

OpenGL ES X11 EFL GStreamer . . . 1.0/2.0

OAL (OEM Adaptation Layer) Telephony GStreamer Sensor System OpenGL ES/EGL Kernel . . . Plug-ins Plug-ins Plug-ins Plug-ins Graphics Driver

Kernel

Memory Graphics Multimedia Power Management Storage Peripheral Management

Block Layer DMA DRM CPUfre Devf Ther Input, Sensor, CMA IOMMU V4L2 … BUF Link1 Link2 q req mal … MMC/Flash

More at http://source.tizen.org/documentation/porting-guide

DRM Link1: http:// elinux.org/images/7/71/Elce11_dae.pdf DRM Link2: http://download.tizen.org/misc/media/conference2012/wednesday/ballroom-c/2012-05-09-1330-1410- 4 the_drm_(direct_rendering_manager)_of_tizen_kernel.pdf tizen.org Tizen Kernel Overview

Core Service Layer

Multimedia Telephony • Memory ManagementSensor FW in TizenSystem FW . . . FW FW – Coupled with Graphics & Multimedia devices. OpenGL ES X11 EFL GStreamer . . . • Graphics & Multimedia devices = DMA1.0/2.0 devices with HUGE buffers

OAL (OEM Adaptation Layer) Telephony GStreamer Sensor System OpenGL ES/EGL Kernel . . . Plug-ins Plug-ins Plug-ins Plug-ins Graphics Driver

Kernel

Memory Graphics Multimedia Power Management Storage Peripheral Management

Block Layer CM IOMM DMA CPUfre Devf Ther Input, Sensor, DRM … V4L2 q req mal … A U BUF Link1 Link2 MMC/Flash

More at http://source.tizen.org/documentation/porting-guide

DRM Link1: http:// elinux.org/images/7/71/Elce11_dae.pdf DRM Link2: http://download.tizen.org/misc/media/conference2012/wednesday/ballroom-c/2012-05-09-1330-1410- 5 the_drm_(direct_rendering_manager)_of_tizen_kernel.pdf tizen.org Tizen Kernel Overview

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 6 Framework Library Adaptation * Images from MS Office clipart and tizen.org Contents

• Tizen Kernel Overview • Memory Management in Tizen Kernel – To Run Tizen OS. • Basic Memory Management Components • Buffer Sharing (UMM/DMABUF) • Buffer Allocation (UMM/DMAAPI) • Buffer Synchronization • Memory Size Optimization

7 tizen.org

Tizen Kernel MM, Graphics (DRM)

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 8 Framework Library Adaptation * Images from MS Office clipart and Samsung tizen.org Graphics: DRM/GEM (Graphics Execution Manager)

Linux

DRM Common Framework

drm_mode_config drm_fb_helper GEM

Connector Crtc Plane drm_framebuffer Encoder

Connector Crtc Plane drm_framebuffer fb_info GEM Encoder

Allocator

Common Specific

9 tizen.org Graphics: DRM/GEM (Graphics Execution Manager)

Linux Graphics ExecutionDRM CommonManager Framework drm_mode_config drm_fb_helper GEM

• FrameworkConnector developed by Intel Crtc Plane drm_framebuffer • To manageEncoder graphics memory

• FrameworkConnector for buffer management. Crtc Plane drm_framebuffer fb_info GEM • AllocationEncoder and sharing.

Allocator

Common Specific

10 tizen.org Graphics: DRM/GEM Allocation

• GEM Allocation steps @ Tizen (Generic)

1. DRM_IOCTL_MODE_CREATE_DUMB • Create GEM object(global) & user GEM handle(per process) • dumb_create() of struct drm_driver • No physical memory allocated. 2. DRM_IOCTL_MODE_MAP_DUMB • Create fake mmap offset of a gem object and relay the object to user • A hash key of the gem object. • dumb_map_offset() of struct drm_driver 3. MMAP • Request mmap based on the hash key as the offset • Create user address space • Setup cache attribute. • Not mapped to physical memory, yet

11 tizen.org Graphics: DRM/GEM Allocation

• GEM Allocation steps @ Tizen (Generic)

4. On-demand Paging – Implement & Register a fault handler that • With a page fault, allocate a page and map the page. – vma->vm_ops->fault = xxx_drm_gem_fault

5. Use!

6. DRM_IOCTL_MODE_DESTROY_DUMB – Remove GEM handle & object – Free memory – Implement dumb_destroy() of struct drm_driver

12 tizen.org Graphics: DRM/GEM Allocation

• GEM Allocation steps @ Tizen ( Only)

1. DRM_IOCTL_EXYNOS_GEM_CREATE • Only use user-desired size and buffer types. • Create gem object(global) & user gem handle(per process) • physical memory allocated.

2. DRM_IOCTL_EXYNOS_GEM_MMAP • Create user address space • Map the user address space to physical memory

• LIBDRM of Exynos uses these APIs, not the generic.

13 tizen.org Graphics: DRM/GEM Sharing

• GEM Sharing @ Tizen • DRM_IOCTL_GEM_FLINK – “I will share this GEM to others.” – Create GEM object name for the given GEM handle • Global key vaue for sharing

• DRM_IOCTL_GEM_OPEN – “I want to use the shared GEM.” – Create GEM handle based on the given GEM object name • DRM_IOCTL_GEM_CLOSE

• You don’t need to implement. It’s already there with DRM.

Process 1 Process 2

GEM GEM GEM Create FLINK OPEN gem handle gem object name gem handle

14 tizen.org Tizen Kernel MM, Multimedia (V4L2/VB2)

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 15 Framework Library Adaptation * Images from MS Office clipart and Samsung tizen.org Multimedia: V4L2/VB2

• Tizen recommends to use V4L2 at Tizen kernel for Multimedia devices • Video input (codec & camera) & Radio • However, as long as the kernel has: – Gstreamer/OpenMAX plugins – A method to share with other F/W via DMABUF of UMM, • Tizen multimedia works.

• If V4L2/VB2 is used, things get easier.

16 tizen.org Tizen Kernel MM, OpenGL/G3D-GPU

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 17 Framework Library Adaptation * Images from MS Office clipart and Samsung tizen.org OpenGL / G3D-GPU

• Most ARM SoC GPUs (MALI, SGX, …) use their own memory manager • E.g., Exynos4412/4210 Tizen Reference Kernel has Mali w/ UMP. – Mali DDK modified to be compatible with UMM-DMABUF.

• If GPU drivers use DRM, it would be great. • (and make them GPL)

18 tizen.org Contents

• Tizen Kernel Overview • Memory Management in Tizen Kernel – To Run Tizen OS. • Basic Memory Management Components • Buffer Sharing (UMM/DMABUF) • Buffer Allocation (UMM/DMAAPI) • Buffer Synchronization • Memory Size Optimization

19 tizen.org

NEED FOR SOMETHING, A Scenario

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 DRM Framework GPU DDK VB2 GEM (G3D)

CameraKernel fetches / Device a video Drivers stream (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

CoreThis scenarioCore External is simplified forOEM presentation. Kernel 20 Framework Not anLibrary actual exampleAdaptation of Tizen * Images from MS Office clipart and Samsung tizen.org NEED FOR SOMETHING, A Scenario Draws its own images

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 DRM Framework GPU DDK VB2 GEM (G3D)

CameraKernel fetches / Device a video Drivers stream (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

CoreThis scenarioCore External is simplified forOEM presentation. Kernel 21 Framework Not anLibrary actual exampleAdaptation of Tizen * Images from MS Office clipart and Samsung tizen.org NEED FOR SOMETHING, A Scenario Draws its own images

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem GPU Merges the two / Puts the merged image

V4L2 DRM Framework GPU DDK VB2 GEM (G3D)

CameraKernel fetches / Device a video Drivers stream (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

CoreThis scenarioCore External is simplified forOEM presentation. Kernel 22 Framework Not anLibrary actual exampleAdaptation of Tizen * Images from MS Office clipart and Samsung tizen.org NEED FOR SOMETHING, A Scenario Draws its own images

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem GPU Merges the two / Puts the merged image

V4L2 DRM Framework GPU DDK VB2 GEM (G3D)

CameraKernel fetches / Device a video Drivers stream (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

DRM-FIMD(display) Output

CoreThis scenarioCore External is simplified forOEM presentation. Kernel 23 Framework Not anLibrary actual exampleAdaptation of Tizen * Images from MS Office clipart and Samsung tizen.org NEED FOR SOMETHING, A Scenario Draws its own images At a GEM buffer Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem GPU Merges the two / Puts the merged image At a VB2V4L2 buffer DRM Framework GPU From GEM + VB2 into its own (e.g., UMP)DDK VB2 GEM (G3D)

CameraKernel fetches / Device a video Drivers stream (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

DRM-FIMD(display) Output

GEM buffer

CoreThis scenarioCore External is simplified forOEM presentation. Kernel 24 Framework Not anLibrary actual exampleAdaptation of Tizen * Images from MS Office clipart and Samsung tizen.org Tizen Kernel MM, UMM

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- A mechanism to share between OpenMAX 2D GPU CameraSrc NEON DRM, V4L2, and others backend Kernel w/o memcpy Userptr Kernel / Core & Subsystem

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 25 Framework Library Adaptation * Images from MS Office clipart and Samsung tizen.org Tizen Kernel MM, UMM

Requirement from Tizen platform and hardware Multimedia FW X Server EFL Open • Different Memory Managers: GEM, VB2, GPU-adhoc, … GL • Share buffers XvImageSink XV Extension X Video Drivers EXA Evas GStreamer 2D GPU PIXMAN • w/o memcpy Libdrm backend Gstreamer- OpenMAX 2D GPU • From andCameraSrc to users NEON backend Kernel • Never expose directly to users (e.g., physical address) Userptr Kernel / Core & Subsystem  UMM DMABUF!

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 26 Framework Library Adaptation * Images from MS Office clipart and Samsung tizen.org Tizen Kernel Memory Management Unified Memory Management (UMM)

• Introduced by Jesse Barker, 2011

• Includes – DMABUF (sharing buffer)

– DMA Mapping API for Allocation. – CMA (Contiguous Memory Allocator) IOMMU (MMU for I/O devs)

27 tizen.org Tizen Kernel MM, UMM DMA Buffer Sharing: DMABUF

• Export – GEM/VB2/… object  DMABUF

• Import – DMABUF  GEM/VB2/… object

• Userspace sees DMABUF as a File Descriptor

28 tizen.org DMA Buffer Sharing: DMABUF, Example Camera  Display

Xvsink Camera App Camera src X server FD Video encoder User space KMS Kernel space DRM GEM V4L2 Codec V4L2 Camera

struct dma_buf

HW Display Codec Camera

Memory

29 tizen.org DMA Buffer Sharing: DMABUF, Example Conceptual Data Flow

Xvsink Camera App Camera src X server FD Video encoder User space KMS Kernel space DRM GEM V4L2 Codec V4L2 Camera

struct dma_buf

HW Display Codec Camera

Memory

30 tizen.org DMABUF Usage Example 1/4

Camera App Xvsink Camera src X server FD Camera App Video encoder User space

1) Request V4L2 camera buffer (U) KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 2) Allocate CMA buffer (K) 3) Request a camera frame at the V4L2 buffer (U) struct dma_buf 4) Store the camera frame & HW Notify user (K) Display Codec Camera Memory 5) Request DMABUF export for the V4L2 camera buffer (U) 6) dma_buf_exporter() (K) - Create DMABUF from V4L2 buffer 7) dma_buf_fd() (K) - Provide FD of the DMABUF to user

31 tizen.org DMABUF Usage Example 1/4

Camera App Xvsink Camera src X server FD Camera App Video encoder User space

1) Request V4L2 camera buffer (U) KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 2) Allocate CMA buffer (K) 3) Request a camera frame at the V4L2 buffer (U) struct dma_buf 4) Store the camera frame & HW Notify user (K) Display Codec Camera Memory 5) Request DMABUF export for the V4L2 camera buffer (U) 6) dma_buf_exporter() (K) - Create DMABUF from V4L2 buffer 7) dma_buf_fd() (K) - Provide FD of the DMABUF to user

32 tizen.org DMABUF Usage Example 1/4

Camera App Xvsink Camera src X server FD Camera App Video encoder User space

1) Request V4L2 camera buffer (U) KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 2) Allocate CMA buffer (K) 3) Request a camera frame at the V4L2 buffer (U) struct dma_buf 4) Store the camera frame & HW Notify user (K) Display Codec Camera Memory 5) Request DMABUF export for the V4L2 camera buffer (U) 6) dma_buf_exporter() (K) - Create DMABUF from V4L2 buffer 7) dma_buf_fd() (K) - Provide FD of the DMABUF to user

33 tizen.org DMABUF Usage Example 2/4

Camera App Xvsink Camera src X server FD Camera App Video encoder User space

8) Request FD->GEM conversion (U) KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 9) dma_buf_get(fd) (K) - Get DMABUF from FD struct dma_buf 10) dma_buf_attach(dma-buf) / dma_buf_map_attachment() (K) HW Display Codec Camera - Get Buffer from DMABUF Memory 11) Import as GEM, send user (K) 12) Request GEM object name (U) 13) Return GEM object name (K) 14) Send GEM object name to X (U)

34 tizen.org DMABUF Usage Example 2/4

Camera App Xvsink Camera src X server FD Camera App Video encoder User space

8) Request FD->GEM conversion (U) KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 9) dma_buf_get(fd) (K) - Get DMABUF from FD struct dma_buf 10) dma_buf_attach(dma-buf) / dma_buf_map_attachment() (K) HW Display Codec Camera - Get Buffer from DMABUF Memory 11) Import as GEM, send user (K) 12) Request GEM object name (U) 13) Return GEM object name (K) 14) Send GEM object name to X (U)

35 tizen.org DMABUF Usage Example 3/4

Camera App Xvsink Camera src

X server FD

Video encoder User space

KMS Kernel space DRM V4L2 Codec V4L2 Camera X server GEM 15) Convert given GEM object name to GEM. Display its content. struct dma_buf (U & K)

HW Display Codec Camera

Memory

36 tizen.org DMABUF Usage Example 3/4

Camera App Xvsink Camera src

X server FD

Video encoder User space

KMS Kernel space DRM V4L2 Codec V4L2 Camera X server GEM 15) Convert given GEM object name to GEM. Display its content. struct dma_buf (U & K)

HW Display Codec Camera

Memory

37 tizen.org DMABUF Usage Example 4/4

Camera App Xvsink Camera src X server FD

Video encoder User space

Close after usage KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 1) “Free” the GEM object (U) 2) Remove reference from the DMABUF (K) struct dma_buf 3) Close(DMABUF-FD) at cam app (U) HW Display Codec Camera 4) (No more reference to DMABUF) Memory Release callback executed (K)

38 tizen.org DMABUF Usage Example 4/4

Camera App Xvsink Camera src X server FD

Video encoder User space

Close after usage KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 1) “Free” the GEM object (U) 2) Remove reference from the DMABUF struct dma_buf 3) Close(DMABUF-FD) at cam app (U) HW Display Codec Camera 4) (No more reference to DMABUF) Memory Release callback executed (K)

39 tizen.org DMABUF Usage Example 4/4

Camera App Xvsink Camera src X server FD

Video encoder User space

Close after usage KMS Kernel space DRM V4L2 Codec V4L2 Camera GEM 1) “Free” the GEM object (U) 2) Remove reference from the DMABUF struct dma_buf 3) Close(DMABUF-FD) at cam app (U) HW Display Codec Camera 4) (No more reference to DMABUF) Memory Release callback executed (K)

40 tizen.org Sorry, Back to DRM/GEM…

• DMABUF Import / Export needs to be added

• DRM_IOCTL_PRIME_HANDLE_TO_FD • Export gem handle into dmabuf fd • DRM_IOCTL_PRIME_FD_TO_HANDLE • Import dmabuf fd into gem handle

41 tizen.org Contents

• Tizen Kernel Overview • Memory Management in Tizen Kernel – To Run Tizen OS. • Basic Memory Management Components • Buffer Sharing (UMM/DMABUF) • Buffer Allocation (UMM/DMAAPI) • Buffer Synchronization • Memory Size Optimization

42 tizen.org

Tizen Kernel Memory Management Unified Memory Management (UMM)

• Introduced by Jesse Barker, 2011

• Includes – DMABUF (sharing buffer)

– DMA Mapping API for Allocation. – CMA (Contiguous Memory Allocator) IOMMU (MMU for I/O devs)

43 tizen.org Tizen Kernel MM, UMM DMA Mapping API @ Tizen Reference

DRM / GEM V4L2 / VB2 1 2 2 2

Common DMA Mapping Framework Driver

CMA Buddy IOMMU

IOMMU

System Memory

1: “Exynos-DRM” provides unified device address space for all DRM devices 2: Each V4L2 device has its own device address space

44 tizen.org Contents

• Tizen Kernel Overview • Memory Management in Tizen Kernel – To Run Tizen OS. • Basic Memory Management Components • Buffer Sharing (UMM/DMABUF) • Buffer Allocation (UMM/DMAAPI) • Buffer Synchronization • Memory Size Optimization

45 tizen.org

Tizen Kernel Memory Management

We Still Have A HUGE Problem!

46 tizen.org Tizen Kernel MM, Buffer Sync.

Multimedia FW X Server EFL Open GL XvImageSink XV Extension X Video Drivers EXA Evas

GStreamer 2D GPU PIXMAN Libdrm backend Gstreamer- OpenMAX Synchronization? 2D GPU CameraSrc NEON backend Kernel Userptr Kernel / Core & Subsystem

V4L2 UMM DRM Framework GPU DDK VB2 DMABUF GEM (G3D)

Kernel / Device Drivers (BSP)

Virtual Post Codec Camera Radio Display HDMI G2D Display Processor

Core Core External OEM Kernel 47 Framework Library Adaptation * Images from MS Office clipart and Samsung tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

Simple Usage Scenario 1. CPU writes to buffer1. 2. CPU tells GPU to do something on buffer1. 3. GPU does something on buffer1. 4. GPU finishes. 5. CPU reads the buffer1 CPU GPU

BUS

DRAM DRAMDRAM

48 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

Simple Usage Scenario 1. CPU writes to buffer1. 2. CPU tells GPU to do something on buffer1. 3. GPU does something on buffer1. 4. GPU finishes. 5. CPU reads the buffer1 CPU GPU

BUS

DRAM DRAMDRAM

49 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

Simple Usage Scenario 1. CPU writes to buffer1. 2. CPU tells GPU to do something on buffer1. 3. GPU does something on buffer1. 4. GPU finishes. 5. CPU reads the buffer1 CPU GPU

BUS

DRAM DRAMDRAM

50 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

Simple Usage Scenario 1. CPU writes to buffer1. 2. CPU tells GPU to do something on buffer1. 3. GPU does something on buffer1. 4. GPU finishes. 5. CPU reads the buffer1 CPU GPU

BUS

DRAM DRAMDRAM

51 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

Simple Usage Scenario 1. CPU writes to buffer1. 2. CPU tells GPU to do something on buffer1. 3. GPU does something on buffer1. 4. GPU finishes. 5. CPU reads the buffer1 CPU GPU

BUS

DRAM DRAMDRAM

52 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

Simple Usage Scenario 1. CPU writes to buffer1. 2. CPU tells GPU to do something on buffer1. 3. GPU does something on buffer1. 4. GPU finishes. 5. CPU reads the buffer1

How to ensure CPU won’t use buffer 1 until step 4? (esp., a user process)

53 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

• What if there are two threads using GPU? – And if the two look at the same buffer? • Not even aware of it?

CPU, CPU, GPU Thread1 Thread2

BUS

DRAM DRAMDRAM

54 tizen.org Tizen Kernel MM, Buffer Sync. What’s the problem?

• What if the two DMA devices (GPU, FIMC) share buffer? – But hard to know it at drivers or user threads.

– FIMC never knows when GPU finishes – FIMC never knows when Threads1 stops using “buffer1” – Threads2 never know when GPU stops using “buffer1”

CPU, CPU, GPU FIMC-IPP Thread1 Thread2 (Image Post Processing)

BUS

DRAM DRAMDRAM

55 tizen.org Tizen Kernel Memory Management Buffer Synchronization

• TGL (Tizen Global Lock) @ Tizen 2.0 – … Let userspace handle the issue … – Kernel patch required.

• Sync Framework (Google) – Jun, 2012. Resources w/o DMABUF (similar with TGL)

• KDS (Kernel Dependency System, ARM) – May 2012 / DMABUF-compatible

• DMA Fence Framework (Canonical/TI) – Aug 2012 / DMABUF-compatible – Work-In-Progress

56 tizen.org Contents

• Tizen Kernel Overview • Memory Management in Tizen Kernel • Memory Size Optimization – To Run w/ More Devices

57 tizen.org Memory Size Optimization: Challenge

• A device with 512MB RAM • Graphics/Multimedia devices want 400MB reserved – No IOMMU. • The userspace wants to run a web browser and HTML5 app!

•REALLY POSSIBLE?

58 tizen.org Memory Size Optimization: Challenge

• A device with 512MB RAM • Graphics/Multimedia devices want 400MB reserved – No IOMMU • The userspace wants to run a web browser and HTML5 app!

•REALLY POSSIBLE? •WHY NOT?

59 tizen.org Minimize H/W Reserved Memory CMA (Contiguous Memory Allocator)

• Camera wants to reserve 400MB. – No IOMMU: too primitive to use paging.

• You don’t use cameras while web browsing.

– Free the 400MB reserved to camera, let userspace use it.

• CMA can do.

60 tizen.org Minimize H/W Reserved Memory

• (Obsolete/Low-cost) ARM SoC’s DMA devices – No IOMMU – Use physically continuous memory chunk. • No paging • Potentially, a lot of memory is wasted.

CPU CPUCPU GPU L2 Cache MMU BUS DRAM Controller DRAM DRAMDRAM

61 tizen.org Minimize H/W Reserved Memory IOMMU

• (Modern/Mid-High-cost) ARM SoC’s DMA devices – IOMMU for DMA devices! – Use paging! • Dynamically allocate & free

CPU CPUCPU GPU L2 Cache MMU IOMMU BUS DRAM Controller DRAM DRAMDRAM

62 tizen.org Minimize H/W Reserved Memory IOMMU

• Exynos4210/4412-based Tizen mobile device – Achieved near-zero reserved memory.

– At the “menuscreen”, only around ~40MB allocated for display and GPU.

– Unfortunately, not included in the Tizen reference kernel. • But easily supported by Vanilla kernel for Tizen reference boards

63 tizen.org Minimize H/W Reserved Memory Results with 1GB RAM Device

-sh-4.1# free total used free shared buffers cached Mem: 1029428 302020 727408 0 16164 133084 -/+ buffers/cache: 152772 876656 Swap: 0 0 0 -sh-4.1# uname -a Linux localhost 3.8.3-00841-gc1981b1 #34 SMP PREEMPT Wed Apr 10 10:24:04 KST 2013 armv7l GNU/Linux

Mostly available to kernel (1005.3 of 1024MB)

Even with Tizen loaded, mostly free to users (856MB of 1GB)

64 tizen.org Memory Size Optimization: Still Hungry?

• Further optimization required for low-budget devices

– Running full features with 512MB?

– Or even < 512MB?

• What about multi-tasking or “background apps”?

65 tizen.org Memory Size Optimization: SWAP

• Swap – zRAM • Used in some Samsung mobile devices successfully

– Or even swap-to-flash?

66 tizen.org Memory Size Optimization: Multitasking / Background Apps

• Limit multitasking & background-app support

• Pseudo multitasking by freezing background apps

67 tizen.org Thank you!