Maxwell - Embedding widgets in WebKit

by Juan Pablo Ugarte

GUADEC Almería, Spain July 6th - 11th Demo

Maxwell is a proof of concept library that lets you embed Gtk widgets in a WebView Use case:

Reutilize Gtk components in webview

Example:

Related articles section in an article view Use case example GtkWidget in a GtkOverlay Embedded Gtk component Etymology

Broadway Gdk backend

=> X Broadway release

=> X Agent 96

=> Agent 86

=> Maxwell Smart How does it work?

Disclaimer: Not a WebKit hacker!

WebKit 2 uses a split process model

- UI/Gtk process

- Web process

- Network process What do we need?

A way to display a widget inside the DOM tree

A way to handle input events properly What do we need?

A way to display a widget inside the DOM tree

A way to handle input events properly Doom tree integration

Simply use a canvas element with the same size as the offscreen widget Doom tree integration

Simply use a canvas element with the same size as the offscreen widget void webkit_web_view_run_javascript (WebKitWebView *web_view, const gchar *script, ...)

/* In JavaScript */ canvas.width = 100; canvas.height = 50; Getting image data void webkit_web_context_register_uri_scheme (WebKitWebContext *context, const gchar *scheme, ...) Getting image data void webkit_web_context_register_uri_scheme (WebKitWebContext *context, const gchar *scheme, ...) Getting image data void webkit_web_context_register_uri_scheme (WebKitWebContext *context, const gchar *scheme, ...) Getting image data

Main Process WebKit Processes GtkWidget:draw ↴ Getting image data

Main Process WebKit Processes GtkWidget:draw ↴ Damage event ⟶ JS draw() ⟶ Getting image data

Main Process WebKit Processes GtkWidget:draw ↴ Damage event ⟶ JS draw() ⟶ GET maxwell:// Getting image data

Main Process WebKit Processes GtkWidget:draw ↴ Damage event ⟶ JS draw() ⟶ GET maxwell:// URI handler ⟵ ↲ Getting image data

Main Process WebKit Processes GtkWidget:draw ↴ Damage event ⟶ JS draw() ⟶ GET maxwell:// URI handler ⟵ ↲ ↳[offscreen]⟶[GdkPixbuf]⟶GIOStream ⟶ Getting image data

Main Process WebKit Processes GtkWidget:draw ↴ Damage event ⟶ JS draw() ⟶ GET maxwell:// URI handler ⟵ ↲ ↳[offscreen]⟶[GdkPixbuf]⟶GIOStream ⟶ [ImageData] ⟶ putImageData() What about Input Events

GdkWindow::pick-embedded-child to let Gtk know which offscreen widget should get the event What about Input Events

GdkWindow::pick-embedded-child to let Gtk know which offscreen widget should get the event getBoundingClientRect() to calculate elements position relative to WebView's viewport What about Input Events

GdkWindow::pick-embedded-child to let Gtk know which offscreen widget should get the event getBoundingClientRect() to calculate elements position relative to WebView's viewport - Scroll and resize event listener

- Mutation Observer API API API API API

Html Convention Building

$ sudo apt-get install meson

$ git clone https://github.com/endlessm/maxwell

$ cd maxwell

$ mkdir _build && cd _build

$ meson

$ ninja

$ sudo ninja install What's next?

- Zero Copy?

Use texture-from-pixmap extension with WebGL api to render widget contents

- Propose it upstream ThankGracias You

Email: [email protected] [email protected]

Blog: blogs.gnome.org/xjuan

IRC: xjuan

http://people.gnome.org/~jpu/docs\ /2018-GUADEC/