LEVERAGING OPEN SOURCE BROWSERS TO OPTIMIZE APPS AND UI PERFORMANCE ON SET-TOP BOXES

Albert Dahan, Co-Founder and CTO, Metrological Wouter van Boesschoten, VP of Technology and Innovation, Metrological

Abstract enable cloud-based apps to run on STBs with Embedded browsers present a host of the speed and consistency of native or local performance challenges for operators. apps while avoiding the costs normally Today, the user experience of enhanced associated with proprietary based browser-based services is largely dependent approaches. on the performance of the browser itself. This paper will outline how to Embedded browsers face set-top box (STB) implement an open sourced based approach CPU power and memory limitations that that enables high performance rendering of affect performance and fidelity. As consumer HTML5 apps and user interfaces. It will demand for personalized OTT offerings discuss the capabilities of the browser in increase (Netflix, Hulu, etc) operators need a detail along with its pros and cons when browser experience that can support a compared to other approaches and its ability greater number of apps and the mechanisms to integrate into the STB ecosystem. in place to search and discover the right apps for each person.

HTML5 creates a standard, but it BROWSER ORIGINS doesn’t solve the issue of having to create a uniform experience on every different device. In the early 90’s as the first building Operators need an in-house browser blocks of the Internet became available there supported by a framework that can run was a need for a program to retrieve, traverse cloud-based services consistently across all and present information from internet devices. Using an open-source approach resources. This ultimately became what we enables operators to leverage the rapid now refer to as an Internet browser. The first increase in innovations coming from widely adopted browser was the NCSA contributors to open-source browsing. Mosiac browser, which was later renamed to Netscape in 1994. Microsoft responded with The solution is to adopt a browser Internet Explorer in 1995 and Opera approach that leverages open-source Software ASA responded with Opera version “WebKit for Wayland” (WPE) components. 2.0 in 1996. Apple’s Safari was first released This new software approach delivers high in 2003 however, the origins of WebKit go performance rendering of HTML5 apps and all the way back to 1998, as part of the KDE next-generation user interfaces, increasing HTML Layout engine (KHTML) and KDE browser performance with a smaller software JavaScript Engine (KJS). The WebKit project footprint, and requires significantly less was started within Apple in 2001 as a fork of memory usage. WPE components enable KHTML and KJS. robust rendering of cloud-based applications and next-generation user interfaces and provide better window management to control multiple applications. Operators can

2016 Spring Technical Forum Proceedings EVOLUTION OF WEBPAGES webpages to start changing from simply offering static information to dynamically In 1992 CERN launched the first providing the latest (server side generated) webpage under the HTML1.0 specification. information such as news and web portals. At the time, a webpage was nothing more This transition from static webpages to web- than text, static images and hyperlinks to applications meant that webpages could now other resources. During the first wave of load dynamic data by making client browser launches the HTML specification originated queries towards (other) Internet evolved at a high pace. Within a few years resources; more data could now be loaded HTML 2.0 (1995), HTML 3.2 with CSS1 from different resources without traversing (1996), and HTML 4.0 with CSS2 and towards a new webpage. With the rise of ECMAScript 1 (1997) were released by the social media in the mid 2000’s websites World Wide Web (W3) consortium. dynamically retrieved data and provided (through plugins such as Flash) the ability to With the general adoption of the playback various media. This in turn, allowed Internet, webpages became less static with users to share their videos, pictures and other server side scripting executed through updates with one another through the technologies such as Common Gateway browser. Webpages were no longer static Interface (CGI) and HyperText Preprocessor single load and click to the next resource (PHP). applications, but interactive applications that retrieved data and updated information as the The Asychronous JavaScript and user stayed within that Internet resource. XML (AJAX) technology paved the road for

Figure 1: Evolution of website development over time

2016 Spring Technical Forum Proceedings As mobile devices became popular in popular. This in turn drove the need for the late 2000’s (ex: iOS, Android) the adaptive streaming and common content concepts of responsive design and single encryption capabilities in the browser. page Model-View-Controller web applications became popular. Various Today’s websites, which are loaded by JavaScript frameworks provided the browsers, are no longer web pages. Even the developer with tooling to build full-blown term ‘web page’ seems to indicate something stateful graphical applications on the browser static. These words depict something that platform. resembles a page in book or text in a newspaper. Today’s web pages do not fit that As mobile devices began to struggle with description at all. These web pages provide browser plugins such as Flash, Video Tag user interaction, dynamically loaded was proposed and early drafts of HTML5 and information (sometimes even real-time), CSS3 started to emerge. Interestingly, the advertisements, two-way communication, drafts of HTML5 came almost along 10 years audio/video streaming and 3D graphics on a after its predecessor (HTML4). While platform that is uniform across multiple browsers started to adopt portions of the devices. Interestingly, this application HTML5 specifications, social media environment isn’t controlled by a single platforms were dominating the Internet. technology company. It is truly an open Following the rise of social media, along platform where the community drives the came the rise of online video streaming, need for changes. Open source is leading the which provided further focus on the way and the browser is by far the most browser’s capability to retrieve and render widely used application platform on the video. With websites such as YouTube Internet. The browser is no longer ‘a program serving millions of videos every day, to retrieve, traverse and present information companies like Netflix, Amazon Prime from Internet resources’. It is a full Video, Hulu started to get immensely application environment.

Figure 2: Website complexity over time 2016 Spring Technical Forum Proceedings

EMBEDDED BROWSER of QT 5.4 this became increasingly harder IMPLEMENTATIONS due to licensing changes by QT. Since QT port isn’t maintained upstream, WebKit is Embedded devices have always been clearly the best choice due to its lightweight a bit of a niche market for browsers. With the nature and BSD v2/LGPL v2 licensing. rise of mobile devices there was no Being maintained by Apple, Adobe, KDE unification on the browser and no massive (graphical desktop environment for UNIX adoption of one or two browsers. Unlike the workstations) and others, the WebKit project desktop market, mobile users do not have a is sure to quickly adopt the latest W3 ‘preferred browser’. Historically, PC users specifications for HTML5.1 and beyond. All install their own browser on their desktop. that was missing was a free and lightweight This is because at least initially operating graphics framework. systems did not come with a latest and greatest browser (i.e. Windows and Internet FUTURE TRENDS OF W3C AND THE explorer). Mobile devices, on the other hand, IMPACT ON EMBEDDED DEVICES are different. They are already bundled with a browser from its respective company (e.g. The shift from static web pages to an iOS comes with Safari, Android with application environment isn’t accidental. The Chrome, etc.) and use the app store W3 consortium started its catch up with other mechanism to keep it up to date (an approach application environments with the start of the later adopted by desktop environments). HTML5 specification. The major highlights in past iterations included video tag, drag and For embedded devices that are not drop, offline apps and Canvas. The new running iOS, Android or Windows Mobile HTML 5.1 specification enables media there is not a lot of choice. There are a few source extensions, encrypted media proprietary solutions available which require extensions, full screen Application Program a license, but often these proprietary Interfaces (APIs), geolocation, Indexed solutions struggle to keep up with the high Database API (IndexedDB) and Web Audio pace of new HTML5 specifications. support.

Through the Chromium project, the However W3 isn’t stopping there. A Blink-based source code was available for closer look at the task forces that are present porting towards embedded devices. This within the W3 reveals that they are planning provides a Chrome-based browser for mobile to add functionality for automotive, mobile devices. However, Blink is built for a desktop devices and TV and broadcasting. There are and requires desktop resources in terms of already W3 drafts/proposals for functionality available memory, Central Processing Unit such as screen orientation, lock screen for (CPU) and graphics power. It runs great on mobile, reading metrics for cars and the expensive embedded hardware, but not all TVAPI for tuning and recording functionality embedded projects can afford 600 dollars’ on broadcasting devices. worth of hardware. W3 is clearly going after these other For quite some time the only license free application environments and with the focus solution was a WebKit port on top of the QT on mobile, cars and TV, the need for an application framework. With the introduction

2016 Spring Technical Forum Proceedings embedded browser is much more important complex. The key is to combine both at the than ever. right place. Use simple solutions where possible and leverage complex optimizations PERFORMANCE CHALLENGES OF that are already out there and vetted by the EMBEDDED BROWSERS community.

Anyone who has developed on Need for a new approach embedded devices will acknowledge that performance is the Achilles’ heel of WebKit for Wayland (WPE) does embedded solutions. Embedded solutions exactly that. Leveraging state-of-the-art continuously balance hardware costs with browser optimizations capabilities provided performance. Like all computer hardware, the by WebKit, such as the JavaScript Core more expensive the hardware components the Fourth-tier optimizing (FTL) JIT compiler, more resources a software program will and combining that with Wayland. Wayland likely have. The more resources a software provides simple, elegant, graphics program can utilize the better the compositing integration between different performance, especially for graphical layers using EGL (interface between Khronos applications. rendering APIs). Wayland started as a replacement for the X Windowing System Those who expect high-end (X11). Wayland is not an implementation but smartphone and gaming console performance a protocol specification between a display out of a 60 dollar device should think again. server and its clients. Its implementations are The actual cost of a smartphone or gaming lightweight with a small footprint. Wayland console that can support cutting edge is primarily focused on performance, code graphics and high performance capabilities maintainability and security. would run almost tenfold of that in terms of costs. The right balance can be hard to find. Integration effort, time to market If the manufacturing costs are too high, the end consumer product will likely be too Creating proprietary solutions for a expensive. A product with functionality that single device with limited requirements will is too limited, even with a small price tag, always be viable. Performance can be will have too many hardware compromises. safeguarded as the complexity of the solution This results in an end product that is within controllable limits. However these underperforms, yielding a negative end user kind of proprietary solutions do not scale experience. very well over multiple devices. Even using open source does not solve the scaling issue. Software matters when considering Using community driven components does the capability of hardware. It won’t make a not necessarily mean it will lower integration 60 dollar device outperform a 600 dollar one. efforts for every device. In other open source However, software should be at a level where browser solutions substantial time is lost on it meets the hardware capabilities. In order to the integration of low-level primitives for the achieve the right performance there are browser such as graphics and input. This several factors, such as complexity and means a full stack integration and effort on optimizations that are crucial. However, both the Software Development Kit complexity and optimizations are almost (SDK)/driver level as well as how the orthogonal, as optimizations can quickly get browser uses those primitives.

2016 Spring Technical Forum Proceedings WebKit For Wayland

WebKit API

WebCore JavaScript Core

DOM CSS HTML Rendering LLINT JIT DFG FTL

Input GStreamer Graphics YARR WTF

View Backends Wayland client Other WaylandOther clients WaylandOther clients Wayland clients

Events Graphics Events

Graphics

Wayland Compositor

Events Composited screen

Kernel

Figure 3: WebKit for Wayland architecture

To solve the issue, Wayland separates Code maintainability these responsibilities. It provides a unified protocol specification for a multi-program The bigger the footprint, the harder it graphical compositing environment and user is to maintain something. Simply put, more input. Simply put, the hardware/SDK lines of code equal more maintenance. Open supplier can ensure that their drivers work source is great, but it’s important to use the with Wayland by using a reference right solution for the problem. The QT implementation (e.g. Weston) without even application framework served a broader, touching a browser. These hardware/SDK bigger scope than just rendering webpages. suppliers can ensure the drivers are This broader scope meant a maintaining a lot operational with a minimal stack. And, the of layers, abstractions and lines of code for a browser can also be validated and tested browser. without doing full-stack integration on the target device. The browser will still need to A browser consists of a piece that be validated when the two are put together, processes, parses and runs the application but the SDK/drivers and the browser (DOM, JS and CSS) and a piece that renders components can be independently validated. graphics. WebKit handles the first series of This independent validation lowers the risk functions. Wayland handles the rendering of integration issues and integration effort, with a much leaner fit. Not only does thus speeding up time to market. Wayland more concisely solve the rendering problem, but it also provides an easy path forward for multiple instances of WebKit and

2016 Spring Technical Forum Proceedings sharing graphical/input resources between is (cross) compiled on a machine that uses other applications through the Wayland human readable source code to turn it into protocol. This functionality is often sought something a machine can understand. That after in any browser solution and is more doesn’t fit the meaning of native here. So important than ever in the embedded market let’s paraphrase native and assume skeptics where multiple video streaming applications mean something that is closer to the hardware are emerging fast. primitives. Often with embedded development, as mentioned above, fewer When selecting an open source layers mean less complexity and could project another requirement is to stay open. actually benefit performance. And that’s Quite often in the past, proprietary solutions correct. Writing code in assembly (if you were based off of something that was open know what you are doing) will always source, but then diverged from open source outperform code that is written in a high level with proprietary or closed modifications. programming language. Writing code in Breaking too far away from the open source assembly would take years, if not decades. architecture means that the two are so far Which is why brilliant engineers in the early apart they can’t be merged anymore. Staying 1970’s came up with generic purpose code compatible and even bug-to-bug programming languages, such as C/C++ and compatible with the upstream project is beyond. paramount. This way, if someone finds an issue they can solve and share it upstream In essence it’s not about writing the and upstream fixes can be easily pulled in. entire application in assembly or low-level Keeping the code upstream compatible is hardware primitives. Let’s focus one level up essential in order to leverage the open source from those primitives like C and OpenGL community. By contributing solutions back and look at the second part of the sentence: into the community everybody benefits. “A native UI implementation is better then a browser”. One can’t compare a browser to a WPE contributes back to the specific UI application written in C. The community and is on average seven days browser by itself doesn’t do anything. It’s behind the main WebKit trunk. The WebKit necessary to write an application that runs for Wayland browser is upstream compatible within the browser. Would an application and will continue to follow the tip of the written straight on top of OpenGL trunk from WebKit. outperform an HTML5 application that runs in the browser? That really depends on the Native or local implementations implementation details of the HTML5 application. Would it outperform a set of Quite often skeptics say, “A native animations in CSS? Not likely, as a browser user interface (UI) implementation is better is very efficient in determining its strategy to then a browser”. Let’s focus on what this render CSS and it has years of experience to implies. back that up. Technologies such as the threaded compositing provide huge benefits First, what is a native UI? in terms of performance that will need to be Technically, native means something that replicated in the C application. This in turn, exists or belongs to one by nature, but in this takes a lot of time (and money) and adds to context that meaning doesn’t apply. Software the complexity.

2016 Spring Technical Forum Proceedings Next, let’s look at complex 3D to note that C and C++ can be very graphics. Will an application that renders unforgiving. Since it has access to low-level complex 3D graphics written in C directly on primitives, making a mistake can have top of OpenGL outperform a HTML5 disastrous effects to the runtime of the application using CSS animations? Yes, application. Due to its low-level nature it without a doubt. CSS is not meant to takes a lot of time to do basic things that are compete against complex 3D graphics. not exposed the same way as in high-level Cascading style sheets are created to style the languages such as JavaScript. This includes text and blocks within the Document Object type casting, threading (service workers/web Model (DOM) tree. In lieu of that, the workers), non-blocking code and all the browser supports doing Canvas and Web utility the browser gives (network stack, Graphics Library (WebGL) straight from the player interfaces, etc.). browser for complex graphics. The browser exposes methods to access those OpenGL If someone makes a mistake in primitives straight from the HTML5 JavaScript the compiler will deal with it. application. You get access to the same low- Mistakes are unavoidable. The JavaScript level interface. In that sense it is as ‘native’, Core in the browser combined with four tiers to use the words of the skeptics, as writing of JIT compilers will kick in as functions are your application straight on top of OpenGL used more frequently within an application. for 3D graphics in C. So if both components The JavaScript Core JIT compilers optimize are using the same APIs there isn’t much the code to a level that is hard to accomplish difference in the one or the other for with one’s own C/C++ implementation. A lot graphics. This makes the argument of of research and development went into these ‘native’ no longer about access to graphical different JIT tiers. Because of this, it’s hard primitives, as both environments have the to beat with your own application. At least same accessibility. not without spending millions of hours on optimization alone. Hours that someone has That leaves the difference of writing already spent and made available within the an application in C versus writing an community as part of the browser. application in the browser in terms of CPU and memory performance. What is the added On top of that an HTML5 application value of a browser over doing an application is a lot easier to maintain with equal amount directly in C? An application in C, if written of functionality in a C/C++ application. Since well, will be faster than anything one can the code does not need to deal with a lot of create in a browser. However the key thing to primitives the code becomes smaller, simpler note here is “if written well”. C and C++ are and less complex. This makes future general-purpose low-level programming maintenance easier. Finding developers is languages, meaning a software developer can easy and often cheaper as it does not require write code in a generic syntax and access very specific expertise that would otherwise low-level APIs. This has benefits over be required in a C/C++ application. It writing the code in assembly. C and C++ still provides equal or better performance due to provide access to those low-level primitives the level of optimizations the browser can such as access to memory and other hardware apply without involving the application APIs. However, writing against low-levels developer. takes a great deal of time and it’s important

2016 Spring Technical Forum Proceedings Back to the question: “What added Much of the development done for value does a browser have over doing your WPE is fed back to the community. Lots of application directly in C?” Well, quite a few: changes originally developed for WPE have ease of large-scale development, code already been provided back to Apple WebKit maintainability, time to market, threaded and G-Streamer. WPE will continue to pull in compositing and built-in performance changes from upstream WebKit. WPE is through its JITs to name just a few. dedicated to continued support and contributions to the open source community. WEBKIT FOR WAYLAND INTEGRATION Because of its open source nature anyone can access the source code. The With the removal of the QT source code is available on a GitHub application framework and providing a lean repository: and mean integration towards Wayland the https://github.com/Metrological/WebKitFor WebKit for Wayland browser can be easily Wayland. integrated in low cost devices. If the device supports a Wayland compositor the Obtaining the source code is just a integration is extremely straightforward. matter of cloning the repository and building However in cases where the System on a for the right target machine (don’t forget to Chip (SoC) vendor does not provide the checkout the Buildroot and OpenEmbedded required support for the Wayland integration repositories). Contributing to WebKit for the WPE ViewBackend can be extended to Wayland is as simple as forking the support direct graphics integration with the repository, make your changes, run the tests required SoC drivers. This can be used as a and create a pull request. The changes will be fallback mechanism in cases where the SoC reviewed by the open source community, and is no longer actively developed on and where if approved, merged into the trunk. That’s it. it is unlikely the hardware drivers will be No license fees, no hours of meetings, no extended or modified to meet the masses of documents and no other overhead. requirements of Wayland. Just code.

WPE comes with reference build CONCLUSION environment recipes for Buildroot and OpenEmbedded for easy adoption within Embedded browsers have always been (existing) build systems. At the writing of the underdog of browsers. Historically, they this article WPE is mostly supported on all were available solely as proprietary or license major hardware platforms, including various restrictive solutions, which didn’t offer a lot Broadcom chipsets, Intel CE chipsets, Nvidia of choice. With HTML5 features growing at gaming platforms and the Raspberry Pi a rapid pace and entering new territories such family. The latter is mainly used for as the mobile, TV and automotive industries, validation of WPE and because the the need for a fast, open, bleeding edge Raspberry Pi is widely adopted within the embedded browser is greater then ever. open source community. This enables the open source community to quickly develop Embedded browsers should be widely and validate using a widely available and available to anyone. Similar to most desktop cheap device. browsers the embedded browser should be

2016 Spring Technical Forum Proceedings free and open source. By combining the latest WPE provides a simple, high performance, WebKit and Wayland integration is simple low footprint, well maintained open source and straightforward. browser. By truly being part of the open source community, everybody is invited to The browser should follow the contribute to the better future of embedded bleeding edge of the W3 standards. By browsers. following the WebKit main trunk closely, all new features can be pulled in with ease.

2016 Spring Technical Forum Proceedings