Faster Webgl Graphics

MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨ OF I !"#$%&'()+,-./012345

Faster WebGL Graphics

MASTER’STHESIS

David Hrachovy´

Brno, Spring 2012 Declaration

Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Advisor: Mgr. Marek Vinkler

ii Acknowledgement

I would like to express my thanks to Mgr. Marek Vinkler, the supervisor of my thesis, for his guidance, suggestions and friendly approach.

iii Abstract

The goal of the thesis was to ﬁnd an efﬁcient way of loading 3d models in WebGL. Several WebGL frameworks for loading and rendering of 3d models were compared based on their performance as well as other capabilities. In addition, a new method for rendering digital content is presented. Compared to simultaneous rendering on multiple HTML canvases, this method enables to reduce GPU memory footprint, reduces initialization time and gives better rendering performance in most of the tested web browsers. Based on the performed tests, a discussion forum, which demonstrates the new method for fast rendering of 3d avatars, was implemented and evaluated.

iv Keywords

WebGL, COLLADA, HTML5, WebGL frameworks, digital asset ﬁle format

v Contents

1 Introduction ...... 2 2 WebGL ...... 4 2.1 OpenGL ...... 4 2.2 WebGL Fundamentals ...... 7 2.3 Web Browsers Support ...... 10 3 The Overlay Canvas ...... 13 3.1 Multiple Canvases ...... 13 3.2 Overlay Canvas Design ...... 16 3.3 Implementation ...... 18 3.4 Testing ...... 22 3.5 Conclusion and Future Improvements ...... 28 4 Model Loading in Frameworks ...... 30 4.1 Relevant Frameworks Selection ...... 30 4.2 Usage Issues ...... 33 4.3 Design of Loading Test ...... 34 4.4 Test Implementation in the Frameworks ...... 41 4.5 Results ...... 46 5 Sample Application ...... 50 5.1 Used Models ...... 50 5.2 Avatars Talk ...... 51 6 Conclusion ...... 54 A Canvas Initialization ...... 66 B The Overlay Canvas Performance ...... 68 C Brief Description of 3d Asset File Formats ...... 70 C.1 3d Asset File Formats Used in our Tests ...... 70 C.2 Other File Formats ...... 71 D Frameworks’ Properties ...... 73 E Model Loading Test Results ...... 76 F Loading Small Models ...... 81 G Contents of the Attachment ...... 83

1 Chapter 1 Introduction

Interactivity in modern web pages is provided mainly by JavaScript 1 code interpreted by a web browser on a client side. The client-side JavaScript code embedded within an HTML2 web page is run by the client computer and allows to alter the document content, interact with the user, and control the web browser [1]. Additional standards are produced to enhance user’s experience. For example the upcoming HTML5 [2] standard proposes in- terfaces for audio and video elements. WebGL [3] is a standard produced by Khronos WebGL Working Group. It provides another HTML Canvas3 rendering context which presents the WebGL API4. Rendering results are displayed on the canvas initialized with WebGL context. Provided API is similar to OpenGL ES 2.0 and the routines may be called directly in JavaScript. Experimental implementation of the API is available in most leading web browsers. Applications with rich 3d content often use 3d assets (3d models) stored in a certain file format. The implementation of loading a 3d model stored in a particular format and the selection of the format itself influences the total time of loading the model. The aim of the thesis is to find an efficient way of loading the models based on existing solutions. This is done by com- paring WebGL frameworks with various file formats. As a result, suitable candidates for web environment are presented. In addition, we examine rendering in multiple canvases and propose a better approach. In the second chapter, a brief description of major WebGL functionality is presented together with examples in JavaScript. The state of the implementation in most popular web browsers is examined as well. The third chapter is dedicated to examination of the proposed Overlay Canvas method. It is a proof of concept demonstration that solves some is-

1. JavaScript is an interpreted programming language with object-oriented capabilities sometimes referred to as ECMAScript [1]. 2. HyperText Markup Language 3. Canvas is an HTML element providing drawable region. 4. Application Programming Interface

2 1. INTRODUCTION sues which were observed in situations where multiple canvases had been used. Implementation of the Overlay Canvas is tested in various browsers with respect to rendering performance and correctness. Benefits and de- tected issues are discussed together with future improvements. The fourth chapter focuses on loading 3d models in various WebGL frameworks. We test selected models, which are stored in common digital asset file format, and other custom file formats provided by the frameworks. Our automated test tool created to simulate a user visiting a website with the model is designed. Loading time of the models in each of the frameworks is measured. The results are then used to implement a sample application using the Overlay Canvas. The sample application implemented in the fifth chapter is a web site that allows registered users to choose their 3d avatar which is shown next to the message they post. The chosen file format is based on the test results from the previous chapter and an additional test focusing on loading several small models. In the last part of the thesis, suitable file formats for fast loading are summarized and the usability of Overlay Canvas in today’s web browsers is discussed.

3 Chapter 2 WebGL

This chapter is devoted to the introduction of WebGL. Its relation to previous OpenGL standards is revisited since many constructs are semantically similar to those of the underlying OpenGL ES 2.0 API. Tight relation of the OpenGL standards is demonstrated in Figure 2.1 [3]. The main part of the

Figure 2.1: WebGL relation to other OpenGL standards. Arrows denote which speciﬁcation was a given standard based on. chapter describes WebGL integration in a web environment which is ac- companied by code samples. The state of WebGL implementation in major web browsers is examined. Selected web browsers are then used in all tests that we performed. Other parts of WebGL such as security, events, detailed differences to OpenGL ES 2.0, and other topics are not covered here, but they can be found in the WebGL speciﬁcation [3].

2.1 OpenGL

OpenGL is a low-level interface to access graphics hardware capabilities. In the desktop world, there are two standard 3d APIs, DirectX and OpenGL. OpenGL is a multiplatform, hardware independent client-server1 architec-

1. Client application running on a host device (e.g. PC) sends data and commands to the server – graphics hardware (graphics chip or card).

4 2. WEBGL ture providing commands to render basic graphical primitives, manipulate textures, framebuffers and other objects. OpenGL is widely used in industry (computer games, CAD applications, modeling tools) as well as in academic community. The standard was deﬁned by Khronos Group2 [4].

2.1.1 OpenGL ES 2.0

OpenGL ES was designed for handheld devices and embedded devices that place different demands on applications than desktops do. They often feature low overall performance that is caused by reduced memory bandwidth, low processor performance, and small battery capacity. Having well established graphics API, Khronos Group deﬁned OpenGL ES 2.0 relative to the OpenGL 2.0 speciﬁcation that emphasizes a programmable 3d graphics pipeline (see Figure 2.2).

Figure 2.2: OpenGL ES 2.0 graphics pipeline

OpenGL ES 2.0 consists of two speciﬁcations: OpenGL ES 2.0 API spec- iﬁcation and OpenGL ES Shading Language (OpenGL ES SL). [5]

2. The Khronos Group is a not-for-proﬁt, member-funded consortium focusing on the creation of royalty-free open standards. OpenGL and OpenCL are some of the standards produced by the consortium.

5 2. WEBGL

OpenGL ES 2.0 API One of the design goals was to create an API suitable for constrained devices. Primarily, the fixed-function pipeline was dropped and programmable rendering pipeline using vertex shader and fragment shader is mandatory. The following text briefly covers differences of OpenGL ES 2.0 API from OpenGL 2.0. Note that the list of removed features is similar to the list of deprecated features in OpenGL 3.0 [4]. Primitives (only sprites, lines and triangles) are drawn using DrawAr- rays (we omit the gl prefix) and DrawElements using vertex arrays. A support for fixed-point numbers was added. Notable features which were removed include: immediate mode, display lists3, color index rendering, coordinate transformation operations and matrix operations4, evaluators, selection and feedback, lighting model, imag- ing subset5, proxy textures, color sum, fog, texture environments and texture functions, texture borders, depth textures, accumulation buffers and alpha test. Note that the majority of the removed functionality was redundant and can be implemented with vertex and fragment shader [5]. The differences include other subtle changes that were not mentioned in the text. For detailed information see the OpenGL ES 2.0 Difference Specification [6]. Since OpenGL ES 2.0 is based on OpenGL, no new hardware is needed. This applies to WebGL as well.6

OpenGL ES Shading Language OpenGL ES Shading Language [7] is based on OpenGL Shading Language version 1.20 with a few changes that will be not dealt with in detail. The following parts were removed: • certain built-in special variables (gl ClipVertex, gl FragDepth), uniform variables (e.g. gl ModelViewMatrix), varying variables and all attributes (e.g. gl Color, gl Normal) • certain implementation dependent constants such as gl MaxLights

3. e.g. GenLists, NewList, CallList 4. This includes manipulating matrix stack (e.g. PushMatrix), Translate and matrix- building functions (e.g. Ortho). The application is responsible for computing necessary ma- trices and loading them as uniform variables in the vertex shader. 5. Pixel rectangles and bitmaps were removed. Only PixelStore command remained because it is necessary for TexImage2D command to allow changing pack/unpack alignment. 6. Based on their blacklists and whitelists of drivers and hardware, web browsers selectively enable or disable support for WebGL.

6 2. WEBGL

• texture1d and shadow samplers

• noise, derivatives and texture3d functions. All of them are available through extensions. In WebGL, only derivatives extension is available [8].

Regarding removed built-in variables, it is up to the application to load appropriate values into user-defined vertex attributes and pass them, if needed, to the fragment shader. Note that there is no implicit conversion between types. In addition, invariant qualifiers were added to enforce the same values in the same expressions within different shaders. For example, say two vertex shaders each set gl Position with the same expression in both shaders, and the input values into that expression are the same when both shaders run. Due to independent compilation of the two shaders, it is possible that the values assigned to gl Position are not exactly the same when the two shaders run. Moreover, precisions qualifiers were introduced, enabling to control variable precision and range. Variables can be declared to have either low, medium, or high precision. These qualifiers declare a minimum range and precision that the implementation must use. Some OpenGL ES implementa- tions can benefit from qualifiers and perform computations faster and with lower power consumption.7 WebGL can exploit the precision qualifiers to significantly improve rendering performance on mobile devices such as iOS 58 [9]. Detailed changes can be found in OpenGL ES SL Specification [7].

2.2 WebGL Fundamentals

2.2.1 WebGL Context

HTML5 speciﬁcation deﬁnes canvas element that provides scripts with a resolution-dependent bitmap canvas, which can be used for interactive rendering. The canvas provides two contexts: 2d (which we will not cover), and webgl 9 that exposes 3d API for drawing on the canvas [2]. Provided the HTML document is loaded and contains canvas element with attribute

7. Note that precision qualiﬁers were introduces also in OpenGL Shading Language 1.30 for code portability with OpenGL ES. Nevertheless, they have no effect on the precisions of variables. 8. iOS is a mobile operating system developed and distributed by Apple Inc. 9. Current web browsers use experimental-webgl context identiﬁer. However, the string is temporary and will eventually change to webgl

7 2. WEBGL

id=”canvasId”, canvas initialization in JavaScript may be performed with getContext function as in the following example:

1 var canvas= document. getElementById( ’canvasId’ ); 2 var gl= canvas. getContext( ’webgl’ , 3 { a n t i a l i a s: false , 4 s t e n c i l: true }); First, object representing the canvas element is obtained. Then, webgl context is requested with optional second parameters list that disables an- tialiasing and enables stencil buffer. If there was an error initializing the context/canvas, null value is returned. Otherwise, WebGLRenderingCon- text object is returned.

2.2.2 API WebGL context object provides methods with almost the same names as OpenGL ES 2.0 commands [3]. For example, say we have the gl object representing WebGL context, the equivalent commands in OpenGL ES 2.0 and WebGL are shown in Table 2.1. WebGL resources (e.g. textures, buffers) are

Code in JavaScript Code in C var buffer = gl.createBuffer (); GLuint buffer; glGenBuffers(1, &buffer); gl.bindBuffer(gl.ARRAY BUFFER, buffer); glBindBuffer(GL ARRAY BUFFER, buffer); var shader = GLuint shader = gl.createShader(gl.FRAGMENT SHADER); glCreateShader(GL FRAGMENT SHADER); shader = gl.shaderSource(shader, ”source”); glShaderSource(shader, 1, &source, NULL);

Table 2.1: Example of equivalent command calls in OpenGL ES 2.0 and WebGL. The type of source variable is char **. gl.createBuffer() returns WebGLBuffer object whose resource can be explicitly released by calling gl.deleteBuffer(buffer).

not identiﬁed by integer object name as in the case of OpenGL. Instead, each resource is represented as a DOM10 object and the resource is guaranteed to exist as long as the object exists. Web browser can, at any point, release the object using the equivalent of a delete call if it is no longer used, for example when the tab with the application is closed. Note that pointers are not deﬁned in JavaScript which results in shorter argument list in the gl.shaderSource call than in the glShaderSource call.

10. Document Object Model is an interface that provides access to content, structure and style of documents (e.g. HTML documents). http://www.w3.org/DOM/

8 2. WEBGL

2.2.3 Typed Arrays

Manipulating raw binary data in JavaScript is slow because of the need for multiple conversions.11 Typed Array specification [10] defines an Ar- rayBuffer type, representing a generic fixed-length binary buffer. To access the buffer as an array of specific types (e.g. 32-bit signed integers), views were defined. Vertex, index, texture, and other data are transferred to the WebGL implementation using the ArrayBuffer and views. Listing 2.1 uses Float32Array view to convert array of numbers to the 32-bit IEEE floating point types array that can be passed to the bufferData command.

1 var v e r t i c e s=[ 2 1 . 0 , 1 . 0 , −1.0 , 3 −1.0 , 1 . 0 , −1.0 , 4 1 . 0 , −1.0 , −1.0 5 ]; 6 // type conversion using view: 7 var float32vertices= new Float32Array(vertices); 8 gl. bufferData(gl. ARRAY BUFFER, float32vertices, gl.STATIC DRAW); Listing 2.1: Using typed arrays for storing vertex data. Note that the buffer object has to be created and binded.

2.2.4 Rendering Loop

Script-based animations are most often performed by scheduling a callback using setTimeout or setInterval function (referred to as timers). A disad- vantage of this approach is that the author of the animation script has no idea what the ideal frequency for updating their animation is. Moreover, it is not guaranteed that the timers will run exactly on schedule12 [2]. Web browser is in better position to determine how many frames per second can be allocated to all of the animations, therefore the requestAnimationFrame was proposed. The web browser can select the framerate to make the animations run as smoothly as possible. If the page is not currently visible, the animation can be automatically throttled so that it does not update often. A basic description of the three functions follows :

11. Where binary data needs to be manipulated, it is often stored as a String and accessed using charCodeAt(), or stored as an Array with conversion to and from base64 for trans- mission [10]. 12. Delays due to CPU load, other tasks, etc, are to be expected.

9 2. WEBGL

• setInterval(callback [, timeout [, arguments... ]]). The callback function is repeatedly executed with the arguments every timeout milliseconds. • setTimeout(callback [, timeout [, arguments... ]]). The callback function is executed with the arguments after timeout milliseconds. • requestAnimationFrame(callback). The function call requests the web browser to execute the callback function before the screen repaint. For further details see HTML5 [2] speciﬁcation (or prior version), and Tim- ing control for script-based animations [11] speciﬁcation. Sample rendering loops are shown in Table 2.2.

setInterval setTimeout requestAnimationFrame function draw() { function draw() { function draw() { // rendering routines // rendering routines // rendering routines } setTimeout(draw, 30) requestAnimationFrame(draw); setInterval (draw, 30); } } draw(); draw();

Table 2.2: Sample rendering loops in WebGL

2.3 Web Browsers Support

The support is examined only in the most widespread web browsers. The Table 2.3 shows web browsers market share in February 2012. We will further deal with only these web browsers since other web browsers have a very small market share. Regarding WebGL support, In- ternet Explorer (IE), who has the highest share, does not support WebGL at all. Moreover, no plans were announced to include the support. Microsoft’s security concerns were expressed in an article WebGL Considered Harm- ful [16]. There is a plug-in for Internet Explorer called IEWebGL [17] that adds support for WebGL but in some areas it shows lower performance. Other web browser vendors provide WebGL support without any plug- in but they may selectively enable or disable support for WebGL according to their whitelist/blacklist13. Table 2.4 summarizes WebGL support. For

13. Whitelist is a list of all GPU/driver/OS combinations for which WebGL is enabled in the web browser. Blacklist is a list of blocked combinations that are not used for rendering in WebGL [18].

10 2. WEBGL

IE1 Google Chrome Firefox Safari Opera StatCounter 35.75 29.84 24.88 6.77 2.02 Net Applications 52.84 18.90 20.92 5.54 1.71 W3Counter 30.7 25.1 24.4 6.5 2.2 Wikipedia 28.30 23.21 21.76 5.82 3.88 1 Internet Explorer

Table 2.3: Web browser market share – February 2012. The values are in percents. The table includes data from all versions of the named web browsers. The data are collected on large samples (e.g. StatCounter analyses 15 billion pageviews per month collected from more than 3 million websites) [12][13][14][15].

IE Google Chrome Firefox Safari Opera Version none 9 4 5.11 12 alpha Our version 9 17.0.963.83 11.0 5.1 12.00 alpha2 1 WebGL is not enabled by default, works only on Mac computer. 2 Build 1306

Table 2.4: WebGL support in web browsers. First row de- notes the lowest possible version where WebGL was enabled. Note that higher versions of the browsers support WebGL as well. The second row shows the latest versions from February 2012 [19][20][21][22].

11 2. WEBGL testing on our Linux machine, we have selected the latest web browsers, namely: Firefox 11, Google Chrome 17 and Opera 12 alpha. If not indicated otherwise, we will refer to them without their version number. Note that Internet Explorer is available only for Microsoft Windows.

12 Chapter 3 The Overlay Canvas

In this chapter, we elaborate on the Overlay Canvas method. The motiva- tion behind the design of Overlay Canvas is related to problems regarding simultaneous rendering of scenes in multiple canvases. As a solution to these problems, we design and implement the Overlay Canvas. Our reference implementation is compared to a method which uses multiple canvases. Finally, the implementation is tested in common web browsers with regard to correctness and performance. Future improvements are proposed as well.

3.1 Multiple Canvases

It is possible to initialize multiple canvases and render separately on each of them. However, we observed certain properties that could be improved. These are discussed in the following subsections.

3.1.1 Shared Resources

A WebGL context [3] is returned by a successful canvas initialization. Every context is thereby connected to exactly one canvas that provides a rendering surface. WebGL objects, including Texture objects and Virtual Buffer Ob- jects, are binded exactly to one WebGL context. Nonetheless, according to a thread [23] at Khronos mailing list, such resources cannot be shared across multiple contexts. Consequently, for each canvas/context, 3d asset data are transferred to GPU1 separately2. Even if the asset data are the same for all the canvases, they are sent to GPU multiple times.

1. Graphics Processing Unit 2. It is recommended to store vertex data on server side (GPU) to avoid transferring the same data each time the draw command is issued [5].

13 3. THE OVERLAY CANVAS

3.1.2 Canvas Initialization Time

The initialization of a canvas is a blocking3 operation which takes a consid- erable amount of time. Figure 3.1 shows initialization times in milliseconds for a various number of canvases. All the tests are performed on a commodity laptop with speciﬁcation in Table 3.1

Time (ms) 4500 Google Chrome 4000 3500 3000 2500 2000 1500 Opera 1000 Firefox 500 #Canvases 1 10 20 30 40 50 60 70 80 90 100

Figure 3.1: Initialization times of multiple canvases. There are selected minimum values of 10 independent runs.

Operating System Linux 3.0.0-16-generic, Ubuntu 11.10 Processor Intel R CoreTMi3-2310M (2,1 GHz, 3MB) Chipset Intel R HM65 Memory 3GB (1+2 GB) DDR3 1333MHz Hard Drive Solid State Disk 64GB Kingston Graphics Card NVIDIA R GeForce with CUDA R GT 520M with 512MB, proprietary drivers 280.13

Table 3.1: Laptop conﬁguration

The initialization time grows steadily with the number of canvases. Opera, despite its alpha version, gives surprisingly good results. On the contrary, Google Chrome’s initialization time can become noticeable if the content, which is rendered on the canvases, takes very small amount of time

3. The execution of following JavaScript commands must wait for the function termina- tion.

14 3. THE OVERLAY CANVAS to render compared to the initializations times. The initialization times may be even higher on slower machines. We should mention initialization times of one canvas that is not displayed in the ﬁgure: 134 ms, 62 ms and 4 ms for Google Chrome, Firefox and Opera, respectively. All collected data can be found in appendix A. In the following part of the text, we discuss our attempt to reduce initialization time by applying lazy initialization and Web Workers.

Lazy Initialization

If we assume that not all the canvases are visible in the web browser’s view- port4, we can initialize the non-visible canvases on demand. We designed and implemented a test that initializes canvases appearing while scrolling the viewport.

Implementation The test is implemented in an HTML ﬁle. At ﬁrst, we initialize visible canvases. Then a rendering loop renders an animated triangle on each of them. In each iteration of the loop, we also check whether a new canvas becomes visible. If so, we initialize it. Note that we do not render to the new canvases because we want to be able to observe a situation which is affected only by the canvas initialization. This way we should be able to initialize non-visible canvases while scrolling the viewport.

Results We evaluated subjective smoothness of scrolling in different browsers. Firefox and Opera give satisfying results. However, in Google Chrome, the animation suffers from tearing when a new canvas gets initialized; the rotation of the triangles stops for a moment.

Initialization Using Web Workers

Web Workers is an HTML5 component which allows to run JavaScript application in a separate worker thread. Non-visible canvases could be initialized on the background and the tearing, observed while scrolling, might disappear. Nevertheless, as there is no access to DOM inside the worker thread, we cannot retrieve a canvas object from the document and initialize it [24]. We tried to bypass these restrictions in the following test by passing a canvas object or WebGL context object to the worker thread.

4. A part of the web page that is currently visible to the user

15 3. THE OVERLAY CANVAS

Implementation The test consists of two files: an HTML file and a JavaScript worker file. In the main file, apart from creating the worker object, we define two variables, canvas and gl:

1 var canvas= document. getElementById( ’canvas0’ ); 2 var gl= canvas. getContext( "experimental-webgl" , { a n t i a l i a s: f a l s e }); These variables were posted separately in a message to the JavaScript worker.

Results As a result, the following error messages were printed in the de- bugging console of the web browsers (the messages are shortened):

• Firefox: The object could not be cloned. Failed to load script: src/- worker.js (nsresult = 0x805303f4)

• Google Chrome: Uncaught Error: SECURITY ERR: DOM Exception 18

• Opera: Uncaught exception: DOMException: DATA CLONE ERR

Thus, we ﬁnd using WebGL in Web Workers not feasible. Since we failed to improve initialization time of multiple canvases, we propose a method that uses only one canvas.

3.2 Overlay Canvas Design

The goal is to mimic rendering on multiple canvases by a single canvas. When using multiple canvases, the space, where the scenes are rendered, is determined by canvases’ size and position. Let us call these areas rendering areas. Let us consider a situation where we removed the canvases but we re- membered their rendering areas and scenes associated with them. The main idea is to use a single canvas for rendering to the rendering areas – the Over- lay Canvas. Figure 3.2 shows rendering on two canvases compared to the Overlay Canvas layered5 above a web page. We can set the order of displayed elements by setting the layer they are displayed in. In both cases, viewer’s visual experience is the same. To achieve the situation presented in the ﬁgure, the following criteria are set:

5. In addition to horizontal and vertical positions, HTML elements lie along a ”z-axis” and are formatted one on top of the other [25].

16 3. THE OVERLAY CANVAS

Rendering areas

Canvas 1

Canvas 2

Overlay Canvas

Figure 3.2: The Overlay Canvas design compared to multiple canvases

• The canvas should be in a layer above the web page in order to render scenes on top of the rendering areas.

• Semi-transparency of the canvas should allow the user to see the content outside of the rendering areas.

• The canvas should always cover entire web browser’s viewport to be able to reach each visible rendering area.

Apart from these goals, the same visual experience with the Overlay Can- vas is also important. In addition, it is assumed that rendering areas are not even partially covered by some other HTML elements. For example, say we had two rendering areas in different layers and one of them would be partially hidden, placing the Overlay Canvas and rendering scenes correctly would not be feasible. The particular situation is shown in Figure 3.3. In order to cover both rendering areas, the Overlay Canvas should be above the canvases. But the canvas would have to keep track of all the objects in relevant layers to be able to render partially covered scene correctly.

17 3. THE OVERLAY CANVAS

Canvas 1 Canvas 2

Figure 3.3: Partially covered canvas

3.3 Implementation

The Overlay Canvas is implemented using JavaScript and CSS6. To set up the Overlay Canvas, placeholders and scene render functions must be defined. Placeholders are arbitrary HTML elements representing rendering areas. Size and position can be obtained from the placeholders. Commands necessary to render a scene are encapsulated in a single function – the scene render function. Prior to executing the function, we execute WebGL view- port7 command with parameters which match the position and size of the particular placeholder we want to render onto. The WebGL viewport command specifies the affine transformation of x and y from normalized device coordinates to window coordinates. In other words, it defines a rectangular area in the drawing buffer where the rendering results are stored. The number of WebGL viewport command calls equals to the number of rendering areas. Even though the same scene is supposed to be rendered into several rendering areas, it would be rendered separately using the We- bGL viewport command and its scene render function. Note that a developer is responsible for removing multiple canvases initialization, setting up placeholders and scene render functions. However, the canvases that are no longer used can be used as placeholders. In our implementation we used neutral8 DIV elements as placeholders since their rectangular shape can represent rendering areas precisely. The following steps are necessary to render multiple scenes in one frame using the Overlay Canvas: 1. Create and initialize the Overlay Canvas – the procedure includes

6. Cascading Style Sheets [25]. CSS is a style sheet language that allows developers and users to attach style (e.g. fonts and spacing) to structured documents (e.g. HTML documents). 7. Do not confuse the WebGL viewport with web browser viewport. 8. By default the element itself is not visible and serves as a container for formatting its content.

18 3. THE OVERLAY CANVAS

ordinary canvas initialization as well as speciﬁc settings that are discussed in below. The result is a WebGL context. 2. Set up scene render functions – functions for rendering each scene must be deﬁned, they are usually stored in an array. 3. Set scene-placeholder relation – it is desirable to know which scene is supposed to be rendered on each of the placeholders. This information can be stored in an associative array where keys are render functions and values are id attributes of an HTML element. Thus, any element providing rectangular area can be used. 4. Render scenes – for each visible placeholder we set up a WebGL viewport, and respective scene render function stored in scene- placeholder relation is then executed. The visibility of the placeholder is determined from the placeholder’s coordinates. Animations consisting of multiple frames are possible. Every time we render scenes, application framebuffer must be cleared and scene data must be recalculated (e.g. objects translation).

The Overlay Canvas Settings Setting Layer Layer (stack level in CSS terminology) in which elements are displayed is deﬁned by z-index CSS property [25]. The Overlay Canvas which has z-index value greater then all of the placeholders will be closer to the user and above the placeholders.

Position We fix the Overlay Canvas position to the position of the web browser’s viewport so that the canvas will always be visible while scrolling the web page. The canvas size is set to fit the entire viewport’s area. How- ever, the viewport size can change if the user resizes the web browser’s window. We register an event listener9 to resize event of the window object. As soon as the event is fired, our handler sets canvas size to match the viewport size.

Click Events The Overlay Canvas covering all elements in the web page prevents the elements from being clicked. If we registered an event listener on any of those elements, it would not capture mouse click events (e.g. mouseup, mouseup, click). CSS property called pointer-events with

9. Method that handles ﬁred events

19 3. THE OVERLAY CANVAS

value set to none can make the canvas ignore click events and pass them to the underlying elements. This property is supported in Firefox as well as Google Chrome but not in Opera. As a workaround in Opera, we could hide the Overlay Canvas when the click event is ﬁred. Coordinates of the mouse click would be used to simulate a new click (document.createEvent function) that would reach the intended element below the canvas. After that the Overlay Canvas could be shown. Nevertheless, this would lead to a noticeable disappearance.

Creating the Overlay Canvas Example Listing 3.1 shows current implementation of the Overlay Canvas creation using jQuery [26] framework.

1 $( "

On the ﬁrst line, the object representing canvas element is created. Lines 2–4 are responsible for setting canvas tag attributes (e.g. canvas size) while lines 5–9 set CSS properties (ﬁxed position). window.innerWidth and window.innerHeight commands denote viewport dimensions. On the 10th line, the canvas is appended to body element in DOM. Finally, on the lines 12 to 15, the handler for resize event is registered. CSS properties that are used are part of CSS 2.1 standard which is supported by all recent web browsers.

Coordinates Transformation

Before rendering a scene we need to call WebGL viewport command with proper coordinates to deﬁne the placement for rendering results. WebGL

20 3. THE OVERLAY CANVAS viewport coordinates need to be calculated because the WebGL window coordinate system is slightly different than the coordinate system in HTML document, as shown in Figure 3.4. Note that even though the drawing viewTop viewLeft elemTop elemLeft

Placeholder elemHeight WebGL viewport

elemWidth viewHeight positionX

Browser's viewport positionY Drawing buﬀer Web page viewWidth

(a) HTML document (b) WebGL window

Figure 3.4: HTML document and WebGL window coordinates. Cir- cles denote the origin of coordinate system. Names indicating size and position are variables used in the implementation. buffer is displayed separately on the right side, it lies exactly on top of the web browser’s viewport. The following equation shows calculation of the WebGL viewport position using variables noted in the ﬁgure: positionX = elemLeft - viewLeft positionY = viewHeight + viewTop - elemTop - elemHeight The size of the WebGL viewport need not to be recalculated because it is the same as the size of the corresponding placeholder. The transformation does not take into account border, margin and padding properties which determine the position of the placeholder’s content area10. We assume that replaced multiple canvases do not have any of these properties set. If they do, the coordinates calculation has to be adjusted.

10. In CSS speciﬁcation, rectangular boxes generated for each element are described by CSS box model. Each box has a content area (e.g., text, an image, etc.) and optional surrounding padding, border, and margin areas [25].

21 3. THE OVERLAY CANVAS

3.4 Testing

We have implemented two tests focused on correctness and performance of the Overlay Canvas. For the following tests, a simple Triangle Scene consisting of two triangles is implemented in a class called Triangle. Red triangle covers exactly one half of the WebGL viewport and triangle with each vertex painted by different color rotates around its axis. Both of them are displayed in ortho- graphic view as shown in Figure 3.5 (a). WebGL context is required to create an instance of the Triangle class. WebGL state variables (e.g. vertex buffer objects, shaders) and procedures responsible for rendering one frame are encapsulated in the Triangle instance. Multiple Triangle instances therefore have their own application logic and do not share resources.

3.4.1 Correctness – Scrolling In our implementation, WebGL viewport coordinates are recalculated in every frame. Because scrolling the web browser’s viewport keeps changing the coordinates, rendered scene position might not match the rendering area position. The following test should help us recognize borders of rendered scene and rendering area while scrolling and resizing the web browser window.

Implementation Our testing HTML ﬁle contains only one placeholder surrounded by black border. Triangle Scene rendered on the placeholder ﬁts the placeholder if and only if the red triangle edges match the placeholder’s border. The ideal situation is shown in Figure 3.5 (a). In the test, the edges match is visually evaluated while scrolling performed by hand. Additionally, to simulate in- tensive computations in the rendering loop, a simple delay function was used. The core of the delay function is looping for a given period of time (milliseconds). The JavaScript code in Listing 3.2 implements the rendering loop. Number of rendered frames is stored in a variable that is incremented in each iteration. The web page height and width was increased using CSS to enable scrolling.

Results Cropped screenshots from web browsers are shown in Figure 3.5. Results are correct in Google Chrome and Opera.

22 3. THE OVERLAY CANVAS

1 function drawScene() { 2 requestAnimationFrame(drawScene); 3 delay(50); // 50 milliseconds delay 4 gl.clear(gl. COLOR BUFFER BIT); // clears color buffer 5 overlay.render(); // renders Triangle scene 6 counter. update(); // increment number of frames 7 } Listing 3.2: Rendering loop used for correctness testing of the Overlay Canvas

(a) Google Chrome (b) Firefox (c) Opera

Figure 3.5: Behavior of Triangle scene during vertical scrolling. Note that the small triangle is rotating thus it has different position in each of the subﬁgures.

23 3. THE OVERLAY CANVAS

However, the rendered scene is shifted in Firefox. When scrolling stopped, the rendered scene returned immediately to the correct position. The size of the shift increased with higher delay time and speed of scrolling. We discovered that the number of frames does not correspond to the number of screen repaints during scrolling phase. In a small test we found out that the number of frames was 128, which is lower than mozPaintCount 11 value 254 in approximately 10 s interval, with delay set to 50 ms. The rendering loop lacks behind the screen repaint frequency resulting in the noticeable shift. We compared the scrolling behavior with rendering without the Overlay Canvas; the canvas was used instead of the placeholder and no WebGL viewport coordinates calculation was needed. No shift occurred in all three web browsers. Regarding Opera in current Overlay Canvas implementation, we found that the placement of requestAnimationFrame function was crucial in the results. When it was moved after line 6 in Listing 3.2, the shift occurred. ANGLE [27] is a project that allows Windows users to run WebGL content by translating OpenGL ES 2.0 API calls to DirectX API calls. On Win- dows 7 we observed that clip coordinates produced by the vertex shader are not correctly transformed to window coordinates if the WebGL viewport is not entirely contained in the drawable area (canvas), i.e. some part of the viewport should be clipped by the canvas border. The issue was observed in web browsers that used ANGLE backend, namely: Firefox 9.0.1, Firefox 12.0 Portable, Google Chrome 18.0. Opera 12.00 beta does not use ANGLE and the results are correct. This defect was already reported [28] in ANGLE bug tracking system12 by the users and confirmed. It is an advantage that Google Chrome and Firefox can be configured to use native OpenGL backend instead of ANGLE13. Nevertheless, all the tests in Google Chrome and Firefox are performed with ANGLE as we expect the bug to be soon fixed. In Figure 3.6, we show rendering results from a testing HTML file attached to the bug tracker issue. Another situation we want to study is the rendering during web browser window resizing. The rendered triangle flickers in Firefox and Opera. This is caused by resizing of the drawing buffer. To avoid flicker-

11. Property of window object indicating the number of screen repaints. It is available only in Firefox. 12. An application that helps developers keep track of reported software bugs 13. Google Chrome can be executed with –use-gl=desktop. In Firefox settings, value of webgl.prefer-native-gl entry must be set to true. The settings can be accessed at address about:conﬁg.

24 3. THE OVERLAY CANVAS

(a) Correct result (bug tracker) (b) Incorrect result (bug tracker)

Figure 3.6: Rendering results with/without ANGLE. In each screenshot, WebGL viewport is called with size of 800 × 800 and position at (0, 0). But the canvas has size of 800 × 600. The top row screenshots are the attached screenshots from the bug tracker. The left screenshot represents the correct result. On the right, WebGL viewport was shrunk. Our rendering results are shown in the bot- tom row and follow the same pattern.

25 3. THE OVERLAY CANVAS ing, we could stop rendering, leaving the last frame visible, and restore the rendering once the resizing is ﬁnished. But correct results cannot be guaranteed since the position of placeholders could for example depend on web browser viewport width. In Google Chrome, the animation slows down. This is visually more acceptable than ﬂickering. However, with 50 ms delay, the scene does not match the rendering area position. This is similar to the shift observed while scrolling in Firefox.

3.4.2 Performance – Grid of Canvases

We compare two approaches – rendering using multiple canvases and using one Overlay Canvas. Comparison is based on frames per seconds (FPS) obtained during a ﬁxed period of time.

Implementation

In the ﬁrst scenario, multiple canvases, total number of 100 canvases with size of 30 × 30, were used. Again, Triangle Scene is instantiated and rendered on each canvas. In Overlay Canvas scenario, 100 placeholders of the same size serve as rendering areas. A single Triangle Scene instance could have been used for rendering to all of the placeholders. This would improve rendering performance of the Overlay Canvas, since for example binding buffers could be done only in the beginning. Nonetheless, in a real situation, scenes will not be necessary the same and binding buffers must be performed prior to rendering into each rendering area. Therefore, separate instances of Triangle Scene were used making the comparison of the two scenarios fair. Figure 3.7 shows 100 canvases in a web browser window. Picture would be the same for the Overlay Canvas scenario. FPS is calculated from approximately 10 seconds interval. Num- ber of frames is acquired from a variable incremented in each rendering loop iteration.

Results

Table 3.2 shows measured frames per second and time needed to render one frame (ms). The Overlay Canvas has signiﬁcantly higher FPS than multiple canvases scenario in Google Chrome and Firefox. In Opera, there is a small performance drop. Since requestAnimationFrame is in the rendering loop, web browsers are expected to render the animation at a regular interval

26 3. THE OVERLAY CANVAS

Figure 3.7: Grid of canvases in Google Chrome

Multiple canvases Overlay Canvas Web browser FPS Frame* FPS Frame* Google Chrome 28 37 190 6 Firefox 14 77 34 30 Opera 53 19 47 22 * time to render one frame (ms)

Table 3.2: Performance of multiple canvases and the Overlay Canvas. The time values are selected minimum of 10 independent runs. The time does not include canvas initialization. Raw data from all the runs are in appendix B.

27 3. THE OVERLAY CANVAS matching the display’s refresh rate [11]. We managed to disable FPS limit in Google Chrome but not in the other web browsers. In the table, maximum reached FPS in Firefox and Opera is a few frames below our display’s refresh rate which is approximately 60 Hz. Thus, the FPS limit should not limit our measured values.

3.5 Conclusion and Future Improvements

Based on the previous tests we sum up the Overlay Canvas properties with respect to multiple canvases approach. The advantages of the Overlay Can- vas are:

• lower initialization time

• better rendering performance in Firefox and Google Chrome

• shared resources on GPU resulting in lower memory footprint

• possible reduction of the same scene rendering using render-to- texture (the description will follow)

In addition, the Overlay Canvas allows the developers to seamlessly inte- grate a WebGL framework. Such possibility is demonstrated in the sample application presented in chapter 5. The disadvantages of the Overlay Can- vas are:

• slightly lower rendering performance in Opera

• issue in Firefox while scrolling the web page

• issue in web browsers using ANGLE backend on Windows

• issues in Google Chrome, Opera and Firefox while resizing web browser window

• elements underlying the Overlay Canvas cannot respond to click events in Opera

We believe most of the issues with various web browsers will be ﬁxed in their future releases. On the other hand, with the development of WebGL standard and its implementation in web browsers, reasons for using the Overlay Canvas might become redundant (shared resources feature could be proposed in the next version of WebGL standard). The following text discusses possible improvements of the Overlay Canvas.

28 3. THE OVERLAY CANVAS

If one particular scene is rendered on multiple rendering areas, the primitives are rasterized multiple times. We could save rendering time by storing one rendering result in a buffer and reusing the buffer. By attaching Tex- ture object as a Framebuffer attachment to Framebuffer object, the scene can be rendered into a texture [5]. For each visible placeholder we can render associated scenes to textures. The textures are then mapped to triangles14 representing rendering areas. Each scene is therefore rendered only once. As a side effect, no WebGL viewport is needed and the ANGLE defect can be avoided. Regarding Firefox scrolling issue, we could avoid the shift by disabling rendering while scrolling. However, disappearance of the scenes would not be very plausible. HTML image elements could be used as placeholders as well as a fallback solution when scrolling is performed. As soon as the scrolling starts, the rendered results could be transferred from color buffer to the application with ReadPixels WebGL command [5].The transferred data could be displayed in the images. The source code of Overlay Canvas and the testing ﬁles are in the attachment.

14. WebGL does not support Quad primitives.

29 Chapter 4 Model Loading in Frameworks

The aim of this chapter is to find an efficient way of loading 3d models. There is a lot of WebGL frameworks that facilitate the use of WebGL, including managing loading and rendering 3d models. We briefly explore the available frameworks and select the relevant ones. The time to load and render various 3d models is measured by a prac- tical test. The test focuses on two types of file formats – common and custom file format. In one test, a common file format supported by 3d content creation suites (e.g. Blender[29]) is used. It is likely that not all frameworks will support one specific format so we select frameworks who support the most common format (e.g. OBJ, COLLADA). Further information on file formats mentioned throughout the text can be found in appendix C. The test files are to be found in the attachment. Other formats, usually based on JSON1, supported by the framework are tested if the frameworks provide tools for converting from a common file format. They may not be as feature-complete as for example COLLADA but they are usually more compact and faster for parsing.

4.1 Relevant Frameworks Selection

In this section we focus on selecting relevant WebGL frameworks that will be tested with respect to model loading. At ﬁrst, we attempt to make a list of available frameworks. Then we determine general requirements that frameworks must meet to make the testing and implementation feasible. The requirements are applied successively in order to present a compre- hensive process of the frameworks’ elimination. Note that the frameworks discussed in the rest of this section are excluded from the test. The ﬁnal list of remaining frameworks is presented at the end of the section.

1. JavaScript Object Notation is a data-interchange text format based on a subset of JavaScript. The JSON data can be essentially loaded into the application as JavaScript code that deﬁnes ordinary JavaScript data structures. http://www.json.org/

30 4. MODEL LOADINGIN FRAMEWORKS

4.1.1 List of Frameworks

Khronos website [30] tries to list all WebGL related activities that are hap- pening on the web, including WebGL frameworks. Based on the website and one investigative blog post [31], we obtained a total number of 30 frameworks. Note that the list cannot be complete nor can it reﬂect current situation. Minor frameworks might be missing (e.g. lack of popularity) and new ones may appear in time. Before we start the inspection of frameworks, we can exclude GTW and EnergizeGL frameworks because their links from the Khronos list are not valid. Moreover, a research using the Google search engine did not give any results which would point to the project website (March 7, 2012). In addition, we excluded smitear [32] – examples in source repository (revision number 285 from October 11, 2011) attempt to use undeﬁned classes and functions. Since GTW, EnergizeGL and smitear cannot be tested, we would like to emphasize that these frameworks will not be dealt with in the text anymore.

4.1.2 Criteria for Selecting Relevant Frameworks

General selection criteria are the following:

• Framework’s use – general-purpose frameworks are preferred because they allow the developer to build a wide range of applications. On the contrary, frameworks designed for one task provide functionality tailored for a speciﬁc domain. OpenWebGlobe [33] deals with processing and browsing geospatial data. GammaJS [34] is designed for development of 2.5d platform games; e.g. game entities can move in 2d space, game levels consist mainly of enemies and platforms.

• License – we require non-restrictive license.

• Ease of integration – we also prefer frameworks which tools can be executed in Linux environment installed on our machine.

Frameworks that do not comply with these criteria are excluded from testing. Frameworks without OpenWebGlobe and GammaJS are listed in appendix in Table D.2. The table describes frameworks’ licenses and the pri- mary programming language they are written in. We can see that the licenses are more or less benevolent except for CopperLicht framework. Cop- perLicht comes with a 3d editor called CopperCube. The editor works for free for 14 days, but it is available only for Windows and OS X.

31 4. MODEL LOADINGIN FRAMEWORKS

Both Oak3D and Inka3D provide plug-ins for Autodesk Maya2 and Autodesk 3ds Max3 modeling software that we do not own. There is no other way to import models. Thus, we additionally excluded CopperLicht, Oak3D and Inka3D. It is worth noting that not all the frameworks are written in JavaScript. WebGL Toolkit (WGT) [35], gwt-g3d [36], GwtGL [37] are Java frameworks built using Google Web Toolkit4. Jax [38] is based on Ruby on Rails 5 following MVC6 pattern.

Frameworks Excluded from Miscellaneous Reasons • JebGL [39] – Java applet designed only to run WebGL applications in browsers lacking WebGL support.

• X3DOM [40] integrates 3d content in a declarative way using X3D ﬁle format. X3D elements are part of HTML DOM tree and can be manipulated by adding, removing, or changing DOM element. This declarative architecture makes the integration into tests rather difﬁ- cult.

Activity Since WebGL is a relatively new technology, it takes time to develop a ro- bust and bug-free framework. Old code base and no response to reported issues indicate that the users will sooner or later face problems. The ﬁnal version of WebGL standard was released on February 10, 2011 meaning that subtle changes in the standard are not reﬂected by frameworks from year 2010 [3]. The most recent releases are shown in appendix in Table D.1. We observed that Curve3D [41] development stopped on June 18, 2010. The last issue is from 20107. SpiderGL [42] has its last issue on Oc- tober 1, 2010 [43] and last revision on August 3, 2010. Additionally, WebGL

2. http://usa.autodesk.com/maya/ 3. http://usa.autodesk.com/3ds-max/ 4. Google Web Toolkit (GWT) allows developers to write client-side applications in Java and deploy them as JavaScript. http://code.google.com/webtoolkit/ 5. Ruby on Rails is a web application framework written Ruby programming language http://rubyonrails.org/ 6. Model-View-Controller 7. It seems that the issue tracker served the author of the framework as a platform for development ideas. No other users posted a bug report. The issue tracker does not always show issue date in a precise format. Relative time is shown instead. This means that when we visited the web page on March 7, 2012, the date indicating the last issue was 2 years ago.

32 4. MODEL LOADINGIN FRAMEWORKS

Toolkit (WGT) was released on December 26, 2009. Thus, we do not test these frameworks.

File Formats File formats supported by the non-excluded frameworks are in appendix in Table D.3. The second column, File formats, describes an ability to load a common file format (e.g. COLLADA, OBJ) describing geometry and ma- terials. Some of the frameworks allow converting 3d asset to their own file format, usually JSON. The third column indicates such a possibility. Note that appropriate tools for conversion to the custom file format are required. From this list, we select the frameworks supporting COLLADA file format together with frameworks supporting custom file format. Even though Kuda supports COLLADA, we do not include it since it uses three.js as an underlying technology. Model loading is implemented in three.js so Kuda together with three.js would be redundant in the test.

Usage Issues In the implementation stage, we encountered issues in certain frameworks regarding model loading. These frameworks are excluded in this section since we want to present a ﬁnal list of frameworks that will be truly tested.

4.2 Usage Issues

SceneJS The official documentation [44] mentions two tools for converting common file format. The first one is SceneJS-PyCollada [45] for conversion from COLLADA. After we reported our issues in the bug tracking system [46][47],[48], we found out that the plug-in is not actively developed [48]. The second converter is called obj2SceneJS and supports only SceneJS version 0.8. However, the current version is 2.0 and the author strongly encourages users to use the latest version which promises better performance [49]. Thus, we do not test the framework.

KickJS Despite the description on the project web page declaring COL- LADA support, the actual implementation does not support textures ref- erenced in COLLADA ﬁle. The textures have to be manually downloaded and mapped to the mesh as can be seen in Model Viewer example on the project web page [50]. Therefore, we do not test the framework.

33 4. MODEL LOADINGIN FRAMEWORKS

Final List List of frameworks together with their properties including tested ﬁle formats are in Table 4.1. Therefore, the ﬁnal list of frameworks for testing is:

Version Date License File formats c3DL 2.2 March 22, 2011 MIT COLLADA CubicVR.js git commit1 March 6, 2012 MIT COLLADA GLGE 0.9 N/A3 New BSD COLLADA O3D r239 July 16, 2011 New BSD JSON2 OSGJS 0.0.7 February 19, 2012 GNU GPL 3 JSON2 three.js r48 March 4, 2012 MIT COLLADA, OpenCTM, Binary2, JSON2 1 Git commit refers to particular code revision but it is too long to ﬁt in the table. Exact commit number can be found in Table D.1 in appendix. 2 Custom ﬁle format 3 Exact release of version 0.9 is unknown. Release date of the previous version can be found in Table D.1 in appendix.

Table 4.1: Properties of selected frameworks c3DL, CubicVR.js, GLGE, O3D, OSGJS and three.js. We excluded 24 frameworks out of 30 for various reasons. Most of the frameworks were out of our interest because of their design goal, early stage of development, old code base, implementation issues, lack of ability to import complex 3d assets, or lack of Linux support. The fact that we chose only 6 frameworks does not mean that there are no other frameworks for building rich and reliable applications supporting complex 3d assets. Other developers might have different needs. For example Inka3D is a good choice for presenting scenes created in Autodesk Maya; GammaJS is suitable for developing platform games; X3DOM allows embedding 3d graphics on a web page in a declarative way without knowl- edge of programming.

4.3 Design of Loading Test

In this section, we present a general idea of measurement of time that is needed to load and render models in the selected frameworks. We begin measuring the time when a web page starts loading and the ﬁnal time is ob-

34 4. MODEL LOADINGIN FRAMEWORKS tained after the ﬁrst complete frame is rendered. All the tests are performed both with and without bandwidth limit in order to simulate network conditions. We designed tests to measure the total time of loading of: • every single model in all frameworks

• all the models in all frameworks Models will be retrieved from a certain 3d asset file format. Each test of a framework and a file format is encapsulated in a single HTML file. To simulate opening a web page by a user, we implemented a script that automatically opens files in a new instance of web browser. Each test is performed ten times to minimize random fluctuations caused by other pro- cesses. Data are collected and processed by a script. This reduces time and potential errors caused by the human factor. We perform the tests locally on a commodity laptop (specifications are in the previous chapter on page 14). Files are not transferred over the network, they are read from a local disk. However, real applications with assets are stored on a server which is accessed with a limited bandwidth and high latency compared to the local storage. These parameters tend to be unstable making the potential test difficult. To simulate the network conditions we used a local server running on the same machine together with a tool for limiting the bandwidth.

4.3.1 Models As seen in Table 4.1, selected file formats include COLLADA, OpenCTM and custom file formats. A small number of models was selected and con- verted to the mentioned formats. Thus, the same model is tested in different formats and frameworks. Note that each custom file format can be tested in one framework. Only a model saved in COLLADA file can be tested in more than one framework. In order to test frameworks with various data, we chose models ranging from around 4 000 triangles to 160 000 triangles.

Selected Models and Their Properties The models we used include the Stanford Bunny from the Stanford 3D Scanning Repository[51] and Hand located in the Utah Animation Repository[52]. These models were used as reference models in many projects in the computer graphics community. The simplest model of all is Duck which is located within GLGE framework repository. Other models are 3D Head Scan by Lee Perry-Smith from Inﬁnite-Realities[53], and

35 4. MODEL LOADINGIN FRAMEWORKS

Walt Disney Head[54]seen in many examples of three.js framework. For the sake of simplicity, we will refer to these models as Bunny, Hand, Duck, HeadScan and Walt. Modiﬁed versions (subsection 4.3.1) of models used for testing are displayed in Figure 4.1.

Figure 4.1: Models rendered with three.jsFrom left to right: Walt, Duck, Hand, HeadScan and Bunny.

Table 4.2 contains a summary of their properties. Note that not all the models have textures. Nonetheless, a large number of triangles is compen- sating for the lack of textures.

Model name Texture size (bytes) Number of triangles Duck 32 504 4 212 Hand 745 159 15 855 HeadScan 2 544 808 17 684 Bunny No texture 20 999 Walt No texture 168 934

Table 4.2: Properties of models

Performed Modifications Original models are from various sources in different file formats. For modifications we used the following tools:

• Blender [29] for resizing, positioning, removing unnecessary objects such as lights, cameras and animations from a scene

• Meshlab [55] for decimation of Bunny mesh. Results generated by Blender decimation tool were not visually satisfying. We decimated

36 4. MODEL LOADINGIN FRAMEWORKS

Bunny mesh because of rendering issues discussed in CubicVR.js test implementation on page 43.

For conversion from one format to another, we used the following tools that are discussed later:

• Blender – conversions of COLLADA, OBJ, PLY ﬁles

• OpenCTM converter [56] and three.js converter discussed on page 46

• O3D Sample COLLADA converter [57], page 45

• osgconv command provided by OpenSceneGraph [58], page 45, for converting to OSGJS format

The process of modifications and conversions with their inputs/outputs is illustrated in Figure 4.2. The original file, located in the upper left corner, is optionally decimated and edited. Final model is stored in Blender, COL- LADA, and OBJ file format for further conversions by other tools. The conversion to custom file formats is performed by a custom batch script written specifically for this task. The models are represented in several files which have different size. Ta- ble 4.3 shows sorted list of sizes of the largest model, Walt. The small sizes at three.js Binary and OpenCTM format come from metadata files. File format with the highest compression ratio is OpenCTM. For low bandwidth, this file format could have significant impact on overall loading time.

File format Size in bytes Compression ratio COLLADA 17 442 535 100.0% O3D JSON 27 688 396 158.7% three.js JSON 9 714 490 55.7% OSGJS JSON 8 932 106 51.2% three.js Binary 5 505 752 + 626 31.6% three.js OpenCTM 572 620 + 690 3.3%

Table 4.3: File size and compression for Walt model in different ﬁle formats

37 4. MODEL LOADINGIN FRAMEWORKS

Figure 4.2: File formats conversion schema. Rectangular nodes represent ﬁle formats. Elliptical nodes represent tools: B – Blender, O3D – O3D Sample Converter, OpenCTM – OpenCTM converter, three.js – three.js converter, OSG – OpenSceneGraph converter. PLY is Polygon File Format. Other ﬁle formats are described in appendix C.

4.3.2 Automated Test Implementation The task is to automatically launch a web browser on given ﬁles and collect the data generated by the loaded web pages. We do not place any special requirements on the script, that is why we chose Ruby programming language that we are already familiar with. The script manages starting a web server, restarting the web browser and processing the collected data. Figure 4.3 illustrates the architecture of the script.

1. At ﬁrst, a web server is started in a separated thread. A list of ﬁles for testing is assembled from the input parameters described below.

2. In the main thread, the script takes one ﬁle out from the list and ex- ecutes a web browser with HTML ﬁle as a parameter. The execution makes main thread wait until the browser is closed.

3. As soon as the test in the browser is ﬁnished, data are sent from

38 4. MODEL LOADINGIN FRAMEWORKS

Figure 4.3: Automated script architecture

the browser to the server listening in the server thread. The data are stored and the running browser is killed.

4. Main thread continues the execution from point 2. Each given ﬁle is tested 10 times. When there are no ﬁles in the list, the script termi- nates.

Raw results are stored in a ﬁle in JSON format. To simplify the process of presenting the data, a script for generating LATEX document containing ta- bles was developed. Since the LATEX script is not related to model loading, we will not describe it here.

Usage The script is called manually from command line. Two parameters are required – command for web browser launch, and a file pattern that is ex- panded by Ruby function to matched files [59]. These files will be successively tested in a web browser whose name was passed as the first parameter. An example of usage: ./test.rb firefox ’∗∗/ model ∗ . html ’ The script traverses current directory recursively executing Firefox with all HTML files starting with model string.

4.3.3 Test Environment Conﬁguration Scene Models are situated in a simple scene. There is a camera and a single point light that are set up in each framework to give similar pictures as in Fig-

39 4. MODEL LOADINGIN FRAMEWORKS ure 4.1. Note that some of the models are not always oriented in the same direction because of different coordinate systems in the frameworks.

Bandwidth

For proper limited bandwidth simulation several tools were tested. We configured trickle [60] to slow down the connection to 300 kB/s. However, 17 MB file was downloaded in 6 seconds meaning that limiting the framework did not work. Charles [61] worked perfectly with Firefox, but the trial version was allowed to run one instance for only 30 minutes. iprelay is another tool for traffic shaping that did not give us satisfying results. Even though we set the bandwidth to 300 kB/s, the real bandwidth was much slower. The solution we used is dummynet [62]designed for testing networking protocols including bandwidth management. After setting the bandwidth to 300 kB/s, the actual speed was more or less what we expected. Actual bandwidth we used in the test is based on real speed tests. Ac- cording to Net Index [63], average speed in the European Union in March 2012 was 13.22 Mb/s. Thus, we would like to simulate a connection with 1600 kB/s bandwidth. We noticed that setting higher bandwidth, 1600 kB/s, with dummynet resulted in real average speed 776 kB/s when downloading 17 MB file. Hav- ing adjusted the bandwidth to 3200 kB/s, the average speed increased to 1.52 MB/s. The tests are performed with this bandwidth configuration. Note that dummynet has to be manually executed. It is not part of the automated script for the loading test.

Web Browser

The tests are performed only in Google Chrome. The reason for using Google Chrome instead of Firefox for testing is the loading time. Loading models in GLGE framework in Google Chrome takes approximately 22 seconds, whereas loading models in Firefox takes approximately 35 seconds. While testing model loading, we noticed that Google Chrome caches model ﬁles between separate runs. Although we set up low bandwidth, high download speed was defying the bandwidth settings. Moreover, simple refreshing a page, instead of opening a ﬁle in a new browser instance, keeps loaded images in a web browser’s cache. There is a browsing mode in Google Chrome called incognito that clears the browser’s cache on exit. Unfortunately, we failed to launch the mode

40 4. MODEL LOADINGIN FRAMEWORKS

from command line because of a Segmentation fault error. Thus, in the test script we clear browser’s cache directory before each browser execution. Such behavior is hardcoded.

Performance Since the tests are performed on a laptop, the performance of the machine could vary during the test. We set GPU adaptive clocking to the performance proﬁle with NVIDA X Server Settings tool. CPU mode was set to maximum performance using Jupiter8. These settings should ensure a relatively stable performance during tests. In addition, we disabled desktop ef- fects that utilize GPU. Detailed information about the system can be found on page 14 in the previous chapter.

4.4 Test Implementation in the Frameworks

In each HTML file, the test is driven by JavaScript that sends the total time of loading to the local server. We define total time of loading as a time given by the difference between end time and start time. Start time is fetched before the framework is initialized. End time is fetched after the first complete frame is rendered. This means that all the geometries and textures are displayed. As soon as the end time is fetched, we stop the rendering and our sendLog function sends the serialized times as GET request to the local server. Typical structure of each HTML file is represented by Listing 4.1.

1 2 3 4 5 var timer = new Timer(); 6 7 8 timer.push(); 9 10 declare debug variables 11 function s t a r t ( ) { 12 start model loading 13 function a r e t e x t u r e s l o a d e d ( ) { ... } 14 function are geometries loaded ( ) { ... } 15 initialize framework 16 rendering loop 17 render frame

~~8. Homepage: http://www.jupiterapplet.org/~~

~~41 4. MODEL LOADINGIN FRAMEWORKS~~

~~18 i f a r e t e x t u r e s loaded() && are geometries loaded ( ) 19 render last frame 20 time.push(); 21 stop rendering 22 sendLog ( ) ; 23~~