Webos LSM for Automotive
Total Page:16
File Type:pdf, Size:1020Kb
webOS LSM for Automotive Jaeyoon Jung LG Electronics webOS Connectivity Content Provider Apps Linux Wayland Web App Manager System Application Manager SDK Enact Luna Surface Manager HTML5 Yocto Chromium GStreamer LS2 Bus system Qt/QML I18N AI 2 LSM: Luna Surface Manager • Window management of application surfaces and System UI • Input event routing from various devices • Virtual Keyboard • Qt/QML Wayland compositor 3 LSM Architecture Control Qt/QML WAM Others Rendering data Native Bus Wayland adaptation Wayland protocol Wayland EGL LSM LSM LSM webOS LSM components bus Luna QML Wayland Compositor QPA DRM / KMS / EVDEV 4 LSM Core Qt/QML WAM Others • QML Wayland Compositor Native – QML application that handles Wayland surfaces Wayland adaptation – Renders system UI elements Wayland protocol and Wayland surfaces together Wayland EGL LSM QML Wayland • QPA Compositor – Platform abstraction plugin in Qt QPA – Depends on hardware DRM / KMS / EVDEV 5 Wayland and Hardware Integration Qt/QML • Wayland WAM Others Native – Display server and client protocol – EGL, OpenGL Wayland adaptation Wayland protocol • Wayland EGL Wayland – EGL layer for buffer sharing EGL LSM between client and server QML Wayland Compositor • Graphics / Input HW QPA – DRM/KMS or SoC specific – Evdev or proprietary input stack DRM / KMS / EVDEV 6 Client Integration Qt/QML • Various Wayland clients WAM Others Native – WAM • Web app manager Wayland adaptation • Provides Wayland integration for web applications Wayland protocol Wayland – Qt/QML EGL • Runs on top of Qtwayland client LSM QML Wayland plugin Compositor – Others • SDL native clients QPA • Clients compatible with standard wayland protocol can run DRM / KMS / EVDEV 7 LSM for Various Products • Challenges – Various GPUs and input devices – Extend Wayland protocol while keeping compatibility to standard protocols – Commonize core compositor features and modularize extensions – Differentiate user interfaces 8 Various GPUs and input devices • Use different QPAs – Proprietary QPA for LG in-house SoC – Open-sourced QPA • eg. EGLFS for DRM-based platforms • Abstraction for Input devices – webOS proprietary input device stack – Evdev for EGLFS 9 Wayland Protocol Extension • WebOSCompositorExtension – Provides Qt plug-in interfaces to Wayland protocol extension – A plug-in that implements a particular Wayland protocol extension can be loaded as needed – Defined as a Qt module QT += weboscompositorextension 10 Core Compositor and Extensions (1/2) • Compositor Base Architecture – Defines the baseline of webOS compositor and common System UI that can be extended or replaced by a product extension as needed – Key architecture • Plug-in interface for native core compositor • QML module interface that overrides the baseline QML views, controllers and types. 11 Core Compositor and Extensions (2/2) • webOS Compositor Plug-in Interface – Plug-in for core class overrides – Extend WebOSCoreCompositor and WebOSCompositorWindow so that it can re-implement methods in a product-specific way • QML Compositor base and Extension – WebOSCompositorBase • Defines the baseline of QML scene based on the common webOS UX • Provides QML types that can be re-defined by a product extension – WebOSCompositor • Implements product-specific layout of views • Re-defines controllers for views including new ones • Re-uses types in the baseline unless re-defined 12 webOS for Automotive • Automotive Experiences – Cluster – In-Vehicle Infotainment in Center stack – Rear Seat Entertainment • Specialties of webOS – Multiple display supports – Quick prototyping of window layouts and system UIs – DRM-enabled media stack – Integration with Content provider applications IVI and RSE 13 LSM for IVI and RSE • Requirements – Multiple displays – Touchscreen per display – Integration with various applications – Customizable window layout and system UI 14 Multiple Displays • Hardware interface App 1 App 1 App 1 App 1 – DRM with multiple connectors • QPA – Platform screens/windows per display • DRM connector – Platform screen – Compositor Platform window Window 0 Window 1 • Compositor QPA – Has multiple compositor windows per display Screen 0 Screen 1 • Platform window – compositor window – One compositor window covers the screen Connector 0 Connector 1 of the display associated DRM 15 Touchscreen per Display • Hardware interface App 1 App 1 App 1 App 1 – /dev/input/eventX • QPA – Delivers touch inputs to an Compositor associated window Window 0 Window 1 • Match touchscreen devices to each window • Send an event in a local coordinate of QPA the target window Screen 0 Screen 1 • Compositor Touchscreen 0 Touchscreen 1 – Handler events per window Linux Input Device 16 Apps and System UI • Integration with Apps – Supports various application types – Media integration • Punch-through • Textured video • Customizable window layout and system UI – Custom WebOSCompositor implementation as per spec – Reuse common features and UI elements from the compositor base 17 Prototype: LSM for IVI 18 Prototype: LSM for RSE 19 Summary • webOS is Linux based and open-sourced platform constructed on the flexible and extensible architecture. • LSM is the Wayland compositor and window manager in webOS that inherits extensibility. • webOS and LSM have specialties for automotive experiences such as IVI and RSE. 20 Find more in http://webosose.org https://github.com/webosose .