Applicaon Management using DirectFB and ilixi Compositor Technology Consulng Company Research, Development & Global Standard Takanari Hayama (IGEL Co.,Ltd./DirectFB integrated media GmbH) Denis Oliver Kropp (DirectFB integrated media GmbH) Hisao Munakata (Renesas Soluons Corp)

Technology Consulng Company IGEL Co.,Ltd. 1 Objecves

• “Lightweight Applicaon Management Framework” – Support exisng soware assets where possible. • No need to modify exisng assets to run on the new framework. – Intuive interface. – Make the best use of available hardware acceleraons. – Small footprint.

September 19-20, 2012 Automove Summit 2012 2 Demo Video

• Demo video

September 19-20, 2012 Automove Linux Summit 2012 3 Demo Environment

• Soware: – DirectFB 1.7.0 pre-release – ilixi – Linux Kernel 3.4.10/2.6.35.9 • Hardawre: – Renesas R-CarH1 • CPU: ARM CoretexA9 Quad-core (1GHz) • RAM: DDR3 1GiB • GPU: PowerVR SGX543MP2

September 19-20, 2012 Automove Linux Summit 2012 4 Architecture Overview

ilixi DirectFB On Screen WebKit Compositor Applicaon Keyboard (App Mgr)

SaWMan LiTE ilixi (Window Mgr (UI Toolkit) (UI Toolkit) Framework)

DirectFB FusionDale FusionSound (Grahpics Middleware) (Service Manager) (Audio Manager)

fusion (IPC) One (IPC)

September 19-20, 2012 Automove Linux Summit 2012 5 What is DirectFB?

• Lightweight graphics middleware. – Small footprint (< 1.5MB core for ARM in version 1.7.0) – Not a server/client architecture like X11. – Licensed under LGPL. • Supported features: – Graphics rendering with hardware acceleraon. • All rendering primives have soware fallbacks. – Opmized soware rendering. • Mul-core support. • NEON/MMX support. – OpenGL/ES2 support. – Mul-process, mul-layer, input handling, windowing etc. – integraon available.

September 19-20, 2012 Automove Linux Summit 2012 6 Key Components

• DirectFB Core – Graphic renderings – Input managements – Frame buffer managements – Graphics hardware managements • Fusion, One, FusionDale – IPC mechanism • SaWMan – Applicaon Management Framework

September 19-20, 2012 Automove Linux Summit 2012 7 DirectFB Architecture

DirectFB Core

gfxdrivers GeneFx systems (H/W Renderer) (S/W Renderer) (H/W manager)

input drivers video/image providers surface pools (key, touch, etc.) (decoder) (resource manager)

September 19-20, 2012 Automove Linux Summit 2012 8 Screen, Layer, Window, and Surface

Window Layers (H/W Specific) App Surface Layer 0 WM Surface Window Window Layer App Composion by Surface Layer 1 Composion by Window Manager Hardware

Surface Screen (LCD etc)

: Surface Blit Layer N App Applicaon can get exclusive access Surface to a layer.

September 19-20, 2012 Automove Linux Summit 2012 9 Rendering in DirectFB

September 19-20, 2012 10 Automove Linux Summit 2012 SaWMan

• Shared Applicaon and Window Manager – Co-developed by Royal Philips and Denis. • SaWMan provides applicaon management framework. – ilixi Compositor runs as an applicaon manager in our scenario. • Applicaon manager can hook into requests made by each applicaon, and override configuraons such as the size of windows.

September 19-20, 2012 Automove Linux Summit 2012 11 What is ilixi?

• A lightweight user interface toolkit for DirectFB – Developed by Tarik Sekmen. – Aims for rich graphical applicaon for embedded Linux system. – Small footprint (~512KB for ARM). – Licensed under LGPLv3.

September 19-20, 2012 Automove Linux Summit 2012 12 ilixi Compositor

• Applicaon Manager developed on top of ilixi toolkit and SaWMan. – Applicaon Launcher – Task switcher • ilixi Compositor treats all applicaons as its widgets. – Applicaons may consist of mulple widgets. – Requires ‘App Definion’ for each applicaon. • Mulple widgets can be shown at the same me, i.e. mulple applicaons can be displayed as well.

September 19-20, 2012 Automove Linux Summit 2012 13 ilixi Compositor

SaWMan

Compositor

AppCompositor #1 Applicaon #1

SurfaceView IDirectFBWindow SurfaceView IDirectFBWindow sends window events

DirectFB

Quoted from hp://ilixi.org/docs/what_is_ilixi.pdf September 19-20, 2012 Automove Linux Summit 2012 14 App Definion Example

OnScreenKeyboard Tarik Sekmen LGPL System 1.0 ilixi_osk APP_OSK, APP_SYSTEM DEP_MOUSE

September 19-20, 2012 Automove Linux Summit 2012 15 Applicaon Flags

Applicaon Type Flag Descripon APP_NO_MAINWINDOW Applicaon uses a surface instead. APP_SPLASH_WINDOW Applicaon has a splash window. APP_ALLOW_WINDOW_CONFIG Applicaon can modify its windows. APP_SURFACE_DONTBLOCK Applicaon is not blocked when hidden. APP_ALLOW_MULTIPLE Applicaon can have mulple instances. APP_NEEDS_CLEAR Background will be cleared with each update. APP_NEEDS_BLENDING AppCompositors will set blending mode for each SurfaceView. APP_AUTO_START Starts applicaon when compositor is inialised. APP_USE_BACK Applicaon can use a Back buon. APP_STATUSBAR Statusbar applicaon. APP_OSK On Screen Keyboard (OSK) applicaon. APP_HOME Home applicaon. APP_SYSTEM System applicaon (not shown on home).

September 19-20, 2012 Automove Linux Summit 2012 16 On Screen Keyboard

September 19-20, 2012 Automove Linux Summit 2012 17 WebKit on DirectFB

• WebKit directly runs on top of

DirectFB ilixi WebKit ilixi_osk – LiTE toolkit is used for now. Compositor – ilixi toolkit support is on its way. – OSK is in ilixi Compositor. • Code is available at: LiTE FusionDale – git://git.direcb.org/git/direcb/ (UI Toolkit) libs/WebKitDFB.git

FusionSound ilixi (for Sound) (UI Toolkit)

DirectFB

September 19-20, 2012 Automove Linux Summit 2012 18 References

• DirectFB – hp://www.direcb.org/ • ilixi – hp://ilixi.org/ – hp://ilixi.org/docs/what_is_ilixi.pdf

September 19-20, 2012 Automove Linux Summit 2012 19 Acknowledgements

• People who made this demo: – Tarik Sekman (Author of ilixi) – Andreas Shimokawa (DirectFB contributor) – Marek Pikarski (DirectFB contributor)

September 19-20, 2012 Automove Linux Summit 2012 20