Cross Platform Development Tools for Mobile Applications, a Performance and Cost Evaluation

Cross Platform Development tools for mobile applications, a performance and cost evaluation

MÅRTEN PÅLSSON

Master’s Thesis at CSC Author: Mårten Pålsson || [email protected] Supervisor CSC: Sten Andersson || [email protected] External Supervisor: Olle Bergling || [email protected] Examiner: Anders Lansner || [email protected]

TRITA xxx yyyy-nn

Abstract

Since the introduction of smart-phones, cellular phones have changed from relatively simple communication devices to becoming more akin to small computers in their own right. The smart-phone market is riddled with different platforms, with the most popular being Google’s Android, and Apple’s iOS. Each of these platforms use their own operating systems and technologies, making the smart-phone eco-system highly fragmented. This leads to problems for companies wishing to target more than one platform, since applications developed for one device will not work on another. To solve this problem several different cross-platform development tools have appeared. These tools aim to provide developers with a solution along the lines of develop once and deploy anywhere. Cross-platform development tools come in many different forms and utilize different technologies to allow applications to work on different platforms. This project has evaluated two of these approaches in order to to provide a suggestion on which is the most suitable for a company wishing to develop applications for multiple platforms. The evaluation has been based on a performance and cost analysis between a web-to-native wrapper and a source code translator. The result of this work is two-fold. Firstly the potential financial gains are based on the skill set of the company in question, and no clear differentiation can be seen between the two approaches. Secondly, while source code translators provide better performance compared to web- to-native wrappers, web-to-native wrappers are more flexible and thus generally a better alternative when developing applications for multiple platforms. Referat

En prestanda- och kostnadsanalys mellan korsplatformsverktyg för mobila applikationer

Mobiltelefoner har utvecklats snabbt de senaste åren. Från att vara relativt simpla verktyg mest inriktade på telefo- ni, så har de utvecklats till små datorer med avancerad funktionalitet och hårdvara. Smart-phones, som de numera kallas, har tagit världen med storm och det finns ett stort antal olika plattformar på marknaden, varav de största är Android och iOS. Varje plattform har ett eget operativsy- stem som är baserad på egna teknologier, vilket har lett till att smart-phone marknaden blivit väldigt fragmente- rad. Detta ställer till problem för företag som vill utveckla applikationer till flera platformar då en applikation som utvecklats till en platform inte fungerar på andra. För att lösa det här problemet så har korsplatformsverktyg trätt fram. Dessa verktyg fungerar på många olika sätt och ut- nyttjar olika teknologier för att tillåta utvecklare att skapa applikationer som går att köra på flera olika plattformar. Det här projektet har jämfört två av dessa olika tillvä- gagångssätt, baserat på en prestanda- och kostnadsanalys, för att kunna föreslå vilket är det lämpligaste alternativet för ett företag som vill utveckla applikationer till flera platformar. Jämförelsen har gjorts mellan web-to-native wrappers och source code translators. Resultatet av detta arbete är tvådealat. Den potentiella finansiella vinsten är densamma för båda tillvägagångssät- ten och är baserad på den kompetens som redan finns in- om företaget. Vad gäller prestanda så presterar source code translators bättre än web-to-native wrappers. Däremot så är web-to-native wrappers mycket mer flexibla vilket gör dem till ett generellt sett bättre alternativ att att använda för att utveckla appliaktioner till flera platformar. Contents

1 Introduction 1 1.1 Background ...... 1 1.1.1 Smart-phone ecosystem ...... 2 1.2 Purpose ...... 3 1.3 Delimitations ...... 3

2 Frame of Reference 5 2.1 Native development ...... 5 2.1.1 Apple iOS ...... 5 2.1.2 Google Android ...... 6 2.2 Cross-platform development ...... 6 2.2.1 Approaches ...... 6 2.2.2 Tools ...... 8

3 Method 13 3.1 Performance evaluation ...... 13 3.1.1 Application Design ...... 14 3.1.2 PhoneGap ...... 15 3.1.3 MoSync ...... 15 3.1.4 Android and iOS ...... 15 3.1.5 Experiment setup ...... 16 3.1.6 Analysis ...... 16 3.2 Cost evaluation ...... 16 3.2.1 Questionnaire ...... 17 3.2.2 Interview ...... 17

4 Results 19 4.1 Performance evaluation ...... 19 4.2 Cost evaluation ...... 21 4.2.1 Interview ...... 21 4.2.2 Questionnaire ...... 22

5 Discussion 25 5.1 Performance evaluation ...... 25 5.1.1 PhoneGap ...... 25 5.1.2 MoSync ...... 26 5.2 Cost evaluation ...... 27

6 Conclusion 29 6.1 Future work ...... 29

7 References 31 Chapter 1

Introduction

1.1 Background

Since the introduction of smart-phones, cellular phones have changed from relatively simple communication devices to becoming more similar to small computers. They have acquired increased processing power and functionality such as GPS, Internet connectivity and cameras. The smart-phone ecosystem is the home of several different platforms with the most popular being Apple’s iOS, Google’s Android and Windows’ Windows Phone [20]. Each of these platforms have their own operating systems, development tools and programming languages. For companies it is desirable to reach as many markets as possible to increase their profits. However, the variety in platforms and the differences between them makes this an expensive prospect for companies developing mobile applications. Since each platform has its own programming language and run-time environment, applications developed for one platform cannot run on another. This means that developing an application for a specific platform requires intimate knowledge of that particular platform. It does not make things any easier that there are several different manufacturers producing devices for the same platforms - such as Samsung and HTC producing devices for Android. Companies wishing to target several platforms will have to acquire the competence to do so (such as programming languages and platform architecture), as well as either dedicate the resources needed for developing the same application in parallel for the different platforms, or spend twice as much time to develop the same application twice. This makes finding a way to develop applications from a single code base, that work on several different platforms, an attractive prospect. Fortunately, there exists several different methods to develop cross-platform applications. One way to do this is to develop a pure web based application that runs in the device browser. However, this approach has some inherent restrictions compared to native applications. Among these are the reliance on constant connection to the Internet, as well as possible restrictions on which device level functionality (such as GPS and camera) the platform’s browser supports [17].

1 CHAPTER 1. INTRODUCTION

To address these problems several tools to develop native cross-platform applications have emerged to oﬀer a solution along the principle of “develop once, deploy everywhere”. There are numerous approaches, and combinations thereof, for these cross-platform development tools (CPTs), but generally the diﬀerent methodologies used can be split into three broad categories; Web-to-native wrappers (an example is PhoneGap), source code translators (examples are XMLVM and MoSync) and run-times (an example is RhoMobile)[17].

1.1.1 Smart-phone ecosystem The smart-phone ecosystem is in a constant state of ﬂux. An example is the rise and fall of Nokia’s operating system Symbian; from a dominating 52.4% market share in 2009 [27], to Q3 2013 when it only held a 7% market share [29]. In 2009 Android was a newcomer to the smart phone market but was already predicted to become a big player in the future. Predictions were that Android would be second biggest actor on the smart phone market by 2013 after Nokia [27]. These predictions were proven to be incorrect as by Q3 2013 Android held a dominant position on the smart phone market with a 57% market share with iOS coming second, and Symbian – as stated above– only a shadow of its former glory.

13 Other Q3 2013 9 Q1 2013 5 Windows Phone 5 BlackBerry OS 6 5 Symbian 7 4 14 iOS 19 54 Android 57 0 10 20 30 40 50 60

Figure 1.1: Mobile OS growth in 2013. Source: (Global Web Index Q3 2013)

As can be seen in Figure 1.1 on mobile OS growth in 2013: Android and iOS are dominant with 57% and 19% market shares respectively, while everyone else either lost shares or stayed the same. Thus it makes sense for a software company developing mobile applications to target Android ﬁrst, and then expand to iOS. If an application were to be developed for both platforms it would reach a total of 76% of the global smart phone market. In statistics provided by Priori Data (2014)[19], on the total amount of downloads in the USA of the 150 most popular applications, it can be seen that only 8% for iOS and 0,3% for Android are paid for. These numbers are nowhere near conclusive, as you would need a complete overlook

2 1.2. PURPOSE of the global application market, but they do give an indication of a trend. That is, that iOS users are more prone to pay for their applications than Android users. This could give developers added incentive to target multiple platforms, since even though Android has the largest user base, the iOS user base proves more lucrative.

1.2 Purpose

The purpose of this report is to evaluate diﬀerent CPTs to ﬁnd which approach is most suitable for a software development company wishing to develop native applications targeting multiple platforms. The evaluation focus will lie on a cost and performance comparison between using CPTs and native development.

1.3 Delimitations

This work will evaluate one web-to-native approach and one source code translator (PhoneGap and MoSync respectively) and the native platforms targeted will be Ap- ple’s iOS and Google’s Android. The software development company that will be used as a basis for the evaluation will be Tekis AB. To further restrict the scope of the project the performance evaluation will be performed on a select group of hardware features; namely geolocation, magnetic ﬁeld sensor (henceforth denominated compass) and ﬁle system access.

Chapter 2

Frame of Reference

2.1 Native development

Native application development refers to writing software for a specific platform, i.e., a specific set of hardware, operating system (OS), processor and its architecture [21]. Some advantages of native application development are: reliable performance, device APIs, platform SDK, established distribution channels, and reliable support[16, 21]. However, because of this specific development strategy, native applications will not run on any but their native platform. Native applications are executable files that run on a device and interact directly with its OS [21]. This means that they can fully access device functionality (e.g., calender and contacts) and hardware (e.g., camera and file system). The native APIs also provide well developed abstractions for user interfaces (UI) and other user interactions and as such provide the best user experience possible [16].

2.1.1 Apple iOS

Apple’s iOS is a proprietary mobile operating system developed and distributed by Apple Inc [31]. It was originally released for iPhone and iPod touch devices in 2007 [31] but has since been extended to the iPad family [12]. iOS is derived from Mac OS X [24] and shares many of its features. As such it is a Unix based system and is the mobile version of the operating system (OS X) that Apple uses on its computers. iOS applications are written in Objective-C [11], and to write and compile iOS applications a Mac and Xcode is needed [13]. Xcode is Apple’s Integrated Develop- ment Environment (IDE). To test applications in Xcode’s internal emulator is free, but to test the application on a device, or to distribute it through the app store, a paid license is needed [14]. The iOS architecture is layered in the following descending order; the Cocoa Touch layer (user interaction), the Media layer, the Core Services layer and the Core OS layer [11]. This abstraction exists to relieve developers of having to communicate

5 CHAPTER 2. FRAME OF REFERENCE with the hardware directly and encapsulates complex features such as threading and sockets [11].

2.1.2 Google Android

Android is a Linux based open source operating system [25] primarily designed for touch screen devices such as tablets, smart-phones and smart watches. Developed by Android Inc and later purchased by Google, Android was unveiled in 2007 [31] together with the founding of the Open Handset Alliance. The Open Handset Alliance is a group of companies led by Google to further open standards for mobile devices [26]. The first smart-phone using the platform, the HTC Dream, was sold in 2008 [31]. Android code is released by Google under the Apache open source license [26]. The Android Open Source Project (led by Google) is responsible for maintaining and further developing the platform [26]. Due to the open source nature of Android, and its popularity, there is a large community of developers writing applications for the platform and extending its functionality [9]. Android applications are written in the Java programming language with the SDK providing support for packaging and compiling C and C++ code [10]. Ap- plications are compiled using the Android SDK and can be developed in Eclipse using the Android Developer Tools plugin, or Google’s own Android Studio [10]. All Android applications execute within their own virtual machine (VM), creating a secure environment [26], ensuring that no applications can access other applications’ data unless specific permission is given. In Android all applications are equal, with no differentiation between core and third party applications [7]. This means that users can fully tailor their phone to their needs and access a broad spectrum of applications and services.

2.2 Cross-platform development

There are several diﬀerent cross-platform development tools (CPT) for developing cross-platform applications, using a variety of approaches.

2.2.1 Approaches

In a large online survey on cross-platform development done by VisionMobile [17] five distinct approaches were defined; Javascript frameworks, App factories, web-to- native wrappers, runtimes and source-code translators. These denominations will henceforth be used since they adequately describe the different approaches. App factories will not be mentioned in this report as they target non-developers, and this report focuses on low level software development.

6 2.2. CROSS-PLATFORM DEVELOPMENT

JavaScript frameworks

Javascript frameworks are tools to create web applications. Since all platforms have an integrated browser with HTML/CSS compatibility the simplest way to reach multiple platforms is through web programming. All users can reach the application from any platform and you can reach a large market quickly. Accessing the web application requires an Internet connection, which is not always desirable depending on the purpose of the application in question, or in areas with bad coverage. Another key problem is that diﬀerent platform browsers implement diﬀrent HTML5 APIs, for example the Android browser supports WebGL while iOS Safari does not [28]. This means that developers will have to be aware of which functionality is or is not supported in each browser. To address the issue of platform browser fragmentation there exist tools such as jQuery Mobile [30]. jQuery Mobile and similar solutions are Javascript frameworks that aim to ease cross-platform web application development by providing a common API across all platforms.

Web-to-native wrappers

Web-to-native wrappers are tools that aim to deliver native applications using web technologies. An application created using a web-to-native wrapper is basically a web application packaged inside of a native application shell. More specifically the applications are written in HTML/CSS and Javascript, and execute within a We- bView on the device. A WebView is a native application containing a chromeless browser, i.e., a browser without navigation capabilities. Since all platforms support exposing their native code to JavaScript code inside WebViews [16], packaged JavaScript API extensions can be used to access device level functionality beyond what the device’s browser is usually capable of, such as; notifications, accelerometer, compass and the file system [28]. To run truly native, and be free from constant internet connection, web-to- native wrappers bundle all necessary resource files into the native application [35]. This allows the WebView to access all the resources it needs such as HTML and JavaScript files from the application itself, and function completely offline on the target device. The most popular example of web-to-native wrappers is PhoneGap, with its JavaScript API being used by other tools (e.g., WorkLight and AppMobi) [17].

Source code translators

Source code translators translate, or cross-compile if you will, the application’s source code into native language code (e.g Objective C for iOS and C# for Win- dows Phone), intermediate bytecode or directly to low level machine code. This code is then either built into an executable binary that can run on the device or combined with a runtime [17]. Examples of source code translators are MoSync and XMLVM. XMLVM translates Java code into intermediate XML code that is then cross-compiled to the native code of the target platform [32].

7 CHAPTER 2. FRAME OF REFERENCE

Runtimes

A runtime is an execution environment that is located between the application and operating system and acts as a cross-platform compatability layer. A runtime shields the application from the diﬀerences in the underlying platforms and contains a common API that the application can call to access device functionality. Run- times vary in their implementation and execute application code on the device using several diﬀerent methods including: virtualization, interpretation, just-in-time compilation and ahead of time compilation [17]. Examples of runtimes are Appcelerator, RhoMobile and Adobe Flex.

2.2.2 Tools PhoneGap

PhoneGap is a web-to-native wrapper. The PhoneGap framework is essentially a framework that allows web applications to function offline and to access device features. It usually consists of two parts; One part that includes the JavaScript business logic that drives the UI and its functionality, and one part that works natively to access and control the device [23]. Since PhoneGap only consists of HTML/CSS, JavaScript and native code, PhoneGap applications can be developed using the IDE of any platform it supports. A PhoneGap applications interface consists of a single WebView which, upon launching the application, loads a HTML start-up page. Control is then passed to the WebView to allow the user to interact with the web application. Javascript code executed from the WebView can pull resources from the resource files bundled with the application, or data from the web just like a normal web application [34]. The UI consists entirely of HTML and CSS code and is specified in the HTML source files [34]. Switching between views and pulling resources as needed is implemented with JavaScript. PhoneGap can make requests for resources and pages asynchronously, and manage the web page content dynamically [34]. This means that it can load resources into the HTML page only as they are needed, instead of loading entire pages, which makes it possible to create applications that are more similar to native applications. Since PhoneGap uses HTML5 it is possible to store all of the needed resource files on the device and thus remove the need for constant connection to the web [34]. This makes the application native since it contains all the resources it needs, e.g., SQL databases, stored on the device. Since PhoneGap bundles all of the required HTML, CSS, and Javascript files, it is not susceptible to the single origin policy (which means that websites are only allowed to run scripts originating from the same site) since the application does not have a domain. Through PhoneGap’s plugin framework it is possible for users to add new features that are not currently available in the PhoneGap API [23]. To write such a plugin two files are needed; a JavaScript file and a native language file. The JavaScript file acts as the interface between the PhoneGap application and the plu-

8 2.2. CROSS-PLATFORM DEVELOPMENT gin as can be seen in Figure 2.1. Plugin functionalities are accessed by the Javascript ﬁle using JavaScript functions. The native language ﬁle includes the native platform code to access the desired features or perform heavy tasks.

Figure 2.1: PhoneGap Architecture. Source: (IBM)

All of the platforms supported by PhoneGap have their own proprietary tools for packaging applications [34]. This means, for example, that they all use different configuration files and folder structures. Also while the calls to the PhoneGap API are the same for all platforms, the internal JavaScript code to access device features differs [34]. This makes it necessary for developers to have an instance of each development environment in order to package and build their application for the different platforms.

MoSync

MoSync SDK is an open source development environment which enables programmers to develop applications for all of the major smart-phone platforms from a single code base. It allows developers to code and build applications through their own IDE, based on the open source project Eclipse, in either C++ or HTML/CSS and JavaScript, or to use the two approaches together to create hybrid applications [2]. Applications created with MoSync are built to their target platform using GCC (GNU Compiler Collection), Pipe-Tool, and the MoSync Packager [4]. The MoSync IDE uses GCC to compile the code to MoSync intermediate language which is then fed into the Pipe-Tool. The Pipe-Tool combines the functions of a resource compiler, linker, and code optimizer. It optimizes the code, builds the code, links it with pre- built MoSync libraries and other resources, and then outputs byte-code or Java

9 CHAPTER 2. FRAME OF REFERENCE which is then ready to be packaged with the appropriate runtime. The Packager then creates a runnable package for the target platform by combining the result with the appropriate runtime, or it produces a project that needs to be recompiled with another tool. The latter case happens when trying to compile an application for Windows Phone or iOS since both of these platforms require compilation in their own native development environments (Visual Studio for WP and Xcode for iOS) to create an installable package for that platform [4]. MoSync has two primary runtime architectures, one for C++ and one for the web-based languages. The design in both cases is very similar and consists mainly of 6 components [4]:

Service layer Incorporates supporting functions such as ﬁle I/O, threading, networking and memory management.

Application framework Holds the runtimes’s entry point. Its functionality varies on diﬀerent platforms but it generally handles platform speciﬁc event management, initialization and destruction.

Syscalls Implements all the basic features required for all platforms such as camera, images, graphics, audio, and networking. It is also responsible for interpreting MoSync resource ﬁles and supporting event management, initialization and destruction.

Resource system Provides functionality for managing resources such as images, sounds and data. It dynamically creates and destroys resources as well as conditionally loading resources depending on the platform, e.g., the device’s screen size.

Internal extensions Speciﬁes the conﬁguration of each OS.

Core Is responsible for the execution of MoSync applications and cooperates with the syscall and resource system. There are three main cores, the VM core, recompiler core and generated core, but they all share a common interface. This allows all other modules to be shared across runtimes without modifica- tion. The VM core is a virtual machine that loads, interprets and executes bytecode. The recompiler core loads MoSync bytecode and recompiles it into platform specific code. The generated core does not interpret or recompile bytecode, instead it runs MoSync intermediate language code precompiled into native code. The different cores are used for different platforms.

MoSync provides several diﬀerent approaches for designing an application UI, based on the developers skillset and preferences. These approaches include designing the UI in HTML/CSS, a NativeUI JavaScript API and a NativeUI C++ library [1]. Developing the UI in HTML/CSS is identical to developing any other web application and provides an identical experience on all platforms [1]. Using the NativeUI

10 2.2. CROSS-PLATFORM DEVELOPMENT

JavaScript API gives the developer access to native widgets such as tabs and but- tons directly from HTML or JavaScript [3]. The NativeUI C++ library provides functions and classes for control of native UI elements from C++ code [5]. Both the JavaScript API and the C++ library map against a widget C API in the SysCall module to produce the native widgets of the target platform [3, 5].

Chapter 3

Method

The purpose of this report is to evaluate diﬀerent approaches used by CPTs, in order to be able to make a recommendation on which is most suitable for a company wishing to develop mobile applications. To do this a performance and cost evaluation has been performed between two diﬀerent open source CPTs and native coding technologies for the iOS and Android mobile operating systems. The two CPTs chosen for the task were PhoneGap and MoSync. PhoneGap was chosen because it is one of the most popular web-to-native CPTs on the market today. Its API is used in several other web-to-native wrapper tools and it has a large user base. MoSync was chosen because it is one of the most popular open source source-code translators and it has a large user base.

3.1 Performance evaluation

According to Corral et al [18] the performance of a mobile application can be mea- sured with a number of parameters. These parameters are execution time, memory usage, and battery consumption. Memory usage and battery consumption are becoming less of a problem with modern devices since smart-phone hardware is becoming more and more potent. They can also be somewhat circumvented by competent programmers by implementing more efficient algorithms, something that is outside of the scope of this project. The application footprint, i.e., the size of the application when installed on the device, can also be used as a parameter, but this is becoming a diminishing problem as smart-phone storage space grows. The storage space available in the original HTC Desire launched in 2010 was 512 MB [15] while the Samsung Note 3 released in 2013 has 32 GB available for the user [33]. This study will evaluate the execution time parameter, since it directly affects the user experience. It also illustrates the differences in performance betweeen different approaches, since it captures the overhead of when an application interacts with the system hardware. Evaluating a performance metric is not as simple as measuring and noting the data. A proper environment must be set up to fairly test the different CPTs on the

13 CHAPTER 3. METHOD two platforms and appropriate methods for data evaluation must be used [22]. To properly understand the impact on performance of the three diﬀerent approaches – PhoneGap, MoSync, and native coding – a set of benchmarks were implemented. To do this a set of software routines were chosen, that utilise different software and hardware resources on a mobile device. These features were implemented in an application identically coded in four diﬀerent versions; one using PhoneGap, one using MoSync and two coded using the native technologies for An- droid and iOS respectively. The aim was to create an environment where these three approaches, viz. PhoneGap, MoSync, and native coding, could be fairly compared.

3.1.1 Application Design The applications consist of a graphical user interface with a set of button used by the controller to call methods that benchmark a given feature. The goal of each method is to make a call to a software or hardware resource from the operating system and wait for a response. The method measures the time from before making the call to after completion and records it in a file for later analysis. In order to make a realistic comparison between the different approaches a selection of features from different system resources were chosen for benchmarking; the compass, geolocation and reading and writing to the file system. The compass benchmark tests the performance of accessing hardware on the device. Geolocation tests both hardware access and network access as the devices location is calculated through using both the GPS and network information. Finally manipulating the file system tests the performance of the application’s data access.

Compass The compass benchmark consists of requesting updates from the device’s magnetic ﬁeld sensor, starting a 10 second timer, and counting the amount of heading updates received from the sensor before the timer expires. This benchmark is designed to measure performance in a situation that simulates real world usage, where a user is more likely to request heading updates in short bursts, to get more accurate heading information, rather than requesting a single update. The benchmark is run 20 times in order to gather enough information to be able to make a useful statistical analysis.

Geolocation The geolocation benchmark is also, like the compass benchmark, designed to test performance during conditions similar to real world use. Updates are requested from the location sensors on the device, a timer is started, and the number of updates received until the timer expires are recorded. Unlike the compass benchmark the geolocation benchmark is run for a full minute. This is because it can take some time for the sensor to connect to the GPS satellites. The applications were programmed to receive updates from the sensor as quickly as possible, taking no

14 3.1. PERFORMANCE EVALUATION consideration to battery consumption, and to discard cached locations to force the continuous acquisition of new ones. This is to fairly test the maximum sensor access performance of each approach, without taking software logic into the equation. This benchmark was run 20 times.

File system

The ﬁle system benchmark is the simplest of the benchmarks, it reads or writes an array of bytes from or to a ﬁle. Two array sizes were used to analyze the added overhead from using the CPT’s. The array sizes used were 32 bytes and 8 kilobytes. Each benchmark was run 100 times.

3.1.2 PhoneGap

The PhoneGap version of the application was coded using JavaScript, HTML and CSS. The different hardware and software routines used by the application were implemented using the PhoneGap API. The first version of the PhoneGap application was coded for Android, but due to the nature of the different operating systems a few changes had to be made for the iOS version. The biggest change was the way the file system was handled. Since iOS does not allow applications to manipulate the file system outside its own environment, which is allowed in Android, changes were made to the paths used by the application. Otherwise only cosmetic changes were made to the code.

3.1.3 MoSync

The MoSync application was coded using C++ and like with the PhoneGap version, MoSync’s own API was used to access device resources. The ﬁrst version of the application was coded for Android and later ported to iOS. However MoSync AB has dropped development of the MoSync SDK due to bankruptcy. This meant that presently the MoSync SDK is not compatible with the newest version (7.1) of iOS. Unfortunately this turn of events was not discovered until late in the implementation phase of the project so there was no time to ﬁnd a solution or workaround. As such the MoSync version of the application was only implemented for Android.

3.1.4 Android and iOS

The native versions of the application were implemented with Java for Android and Objective-C for iOS. The software routines were implemented according to best practice as proposed by the respective platforms.

15 CHAPTER 3. METHOD

3.1.5 Experiment setup

Each of the applications were run on a real mobile device, a Samsung Galaxy S3 for the Android versions and an iPad 2 for the iOS versions. Each application can make system calls to access speciﬁc resources on the device. Before each system call a time-stamp is taken and another directly after completion. This ensures that the whole time span from request to acquisition is taken into account. Since milliseconds is the highest resolution available in JavaScript this is the resolution chosen for the experiment.

3.1.6 Analysis

The analysis on the performance evaluation, between the diﬀerent approaches, was based on the recommendations proposed by Fleming and Wallace [22]. First the results must be normalized to a known machine in order to correctly represent relative system performance. Then the geometric mean is calculated to average the normalized numbers. These results can then be compared and used to draw conclusions. Java was considered to be the known machine on the Android platform since it is the native environment, in the same way Objective-C was chosen as the known machine for iOS.

3.2 Cost evaluation

For a company not already developing mobile applications, adapting CPTs contra using native coding techniques to develop applications would have different financial impacts. To analyze this difference, the demands of the different approaches, and what it would take for a company to adopt them, must be understood. Since the different approaches use different programming languages and (in the case of MoSync, Android and iOS) require different development environments, different skill sets within a company directly affects the cost of adapting any of the approaches. To find out how these different approaches would affect Tekis AB a questionnaire was designed and distributed among the company’s developers in order to find out what form of programming skills and experience Tekis contains. Another important factor is how the company tackles new projects and acquires the required knowledge for them. Accordingly an interview was held with the head developer Pär Gillander. The results of these two investigations were then combined and analyzed to give an estimation of the potential costs of using the different CPTs compared to native development techniques to produce mobile applications.

16 3.2. COST EVALUATION

3.2.1 Questionnaire The following questionnaire was distributed to the software developers at Tekis AB. Each part of the multiple part questions were to be answered on a scale from 1 to 5.

• How great is your skill in the following programming languages?

– C – C++ – Objective-C – Java – JavaScript

• How much experience do you have with the following development environments?

– Eclipse – Xcode – Visual Studio

• How much experience do you have developing software for the following mobile platforms?

– Android – iOS

3.2.2 Interview The interview was an open interview on how Tekis approaches new projects, or- ganizes their development teams, and acquires new knowledge. A summary of the interview can be found in section 4.2.

Chapter 4

Results

4.1 Performance evaluation

This section includes three tables, table 4.1, Table 4.2 and Table 4.3, showing the results of the performance benchmarks. The arithmetic mean and standard deviation are included to show the distribution of the results. The results for the geolocation and compass benchmarks are shown in number of updates per time interval (as described in the method) and the results for the other benchmarks are given in milliseconds. The geometric mean for each benchmark has been normalized to the known machine of the platform. As such Android and iOS has a geometric mean value of 1.

Benchmark: Geolocation Compass Android native 127.25 1885.95 Android PhoneGap 12.25 4828.15 Arithmetic mean (updates) Android Mosync 60.95 46.35 iOS native 62 364.2 iOS PhoneGap 62.35 2073.65 Android native 4.423 39.096 Android PhoneGap 3.210 41.010 Standard deviation (updates) Android Mosync 1.932 1.755 iOS native 0 1.196 iOS PhoneGap 0.875 16.359 Android native 1 1 Android PhoneGap 0.093 2.561 Geometric mean (normalized) Android Mosync 0.479 0.025 iOS native 1 1 iOS PhoneGap 1.006 5.694

Table 4.1: Compass and geolocation benchmark

19 CHAPTER 4. RESULTS

Benchmark: Read 32B Read 8kB Android native 0.034 0.342 Android PhoneGap 15.106 19.048 Arithmetic mean (ms) Android Mosync 0.081 0.083 iOS native 0.420 0.666 iOS PhoneGap 2.435 3.219 Android native 0.014 0.892 Android PhoneGap 1.702 2.26 Standard deviation (ms) Android Mosync 0.016 0.017 iOS native 0.044 0.062 iOS PhoneGap 0.491 0.327 Android native 1 1 Android PhoneGap 456.749 95.755 Geometric mean (normalized) Android Mosync 2.431 0.414 iOS native 1 1 iOS PhoneGap 5.735 4.832

Table 4.2: Reading from a ﬁle benchmarks

Benchmark: Write 32B Write 8kB Android native 0.081 0.378 Android PhoneGap 15.463 18.289 Arithmetic mean (ms) Android Mosync 0.069 0.253 iOS native 10.609 10.787 iOS PhoneGap 8.352 9.15 Android native 0.019 0.181 Android PhoneGap 2.004 1.962 Standard deviation (ms) Android Mosync 0.012 0.476 iOS native 0.895 0.757 iOS PhoneGap 1.035 1.068 Android native 1 1 Android PhoneGap 192.168 50.643 Geometric mean (normalized) Android Mosync 0.855 0.463 iOS native 1 1 iOS PhoneGap 0.785 0.844

Table 4.3: Writing to a ﬁle benchmarks

20 4.2. COST EVALUATION

4.2 Cost evaluation

This section includes a summary of the interview with Pär Gillander and of the data received from the questionnaire. The results from the questionnaire are displayed in bar charts organised by question.

4.2.1 Interview How much does educating a co-worker cost? A rough estimation is that it takes about 200 work-hours before someone can start delivering useful results when starting with a new system. Even more before he/she delivers sustainable results, so about one months worth of wages. This scales linearly so it takes about 400 work-hours for two people to start to deliver results.

How much time does it take? This is very hard to say since it depends on the individual. Last year two new developers were hired for a new project and it took them six weeks to produce a ﬁnished product. It was not ready for distribution but it was functional and relatively stable. Generally speaking it does not take very long for a developer to come up to speed in a new area. Tekis uses a system where you learn while you work instead of taking traditional courses. If someone does take a course in some area they are expected to be able to work with the new task and produce results when they are ﬁnished. One thing they are trying however are on line courses. This allows self-education during projects and for employees to adapt their own education to their personal needs.

Projects When a project is started that requires new knowledge the standard approach is to hire a consultant that is an expert within the required area. The consultant then works with the development team during the life time of the project to provide support, know-how and educate the team. In this way the developers are edu- cated on the new system/technology and the company acquires a very productive team member. If it turns out that the new activity has a more long term place in Tekis’ operation an expert is employed full time instead since the consultants expert knowledge disappears as soon as their contract expires.

21 CHAPTER 4. RESULTS

4.2.2 Questionnaire Results from the questionnaire can be found in figure 4.1, figure 4.2, and figure 4.3

(a) C (Scale 1 - 5) (b) C++ (Scale 1 - 5)

(e) JavaScript (Scale 1 - 5) (f) HTML/CSS (Scale 1 - 5)

Figure 4.1: How great is your skill in the following programming languages?

22 4.2. COST EVALUATION

(a) Eclipse (Scale 1 - 5) (b) Xcode (Scale 1 - 5)

Figure 4.2: How much experience do you have with the following development environments?

23 CHAPTER 4. RESULTS

(a) Android (Scale 1 - 5) (b) iOS (Xcode 1 - 5)

Figure 4.3: How much experience do you have developing software for the following mobile platforms?

24 Chapter 5

Discussion

5.1 Performance evaluation

5.1.1 PhoneGap

As can be seen in Table 4.1, Table 4.2 and Table 4.3 PhoneGap performs the worst in almost every benchmark. To understand the difference in performance compared to the other approaches, the underlying structure of resource calls were analyzed. Whereas native coding technologies use native methods to call resources directly, JavaScript can only access system resources through an execution path that implements at least one callback. To call native functionality, through the PhoneGap API, a JavaScript class in the WebView is called. This generates a message that is caught by the PhoneGap native engine. The native engine executes the native method and passes the result back to the specified JavaScript callback method via a string message. This introduces overhead for calling methods since you have to wait for the resource call to traverse the callback path, and for the result to reach the original caller. The only benchmarks where PhoneGap is the fastest is in writing to the file system on iOS, Table 4.3, and using the compass on both platforms, Table 4.1. It is interesting to note that while native Android is faster than iOS in the file system benchmarks, the Android version of PhoneGap is much slower than the iOS version, as can be seen in Table 4.2 and Table 4.3. One reason for this could be the way resources are accessed on the different systems. On iOS, calls to access resources are highly restricted and usually only one alternative is provided, the way the iOS developers decide is the best. On Android, however, there sometimes exist several different ways to do the same thing, not all of which are equivalent regarding performance. This means that on iOS the relative performance, represented by the geometric mean, might generally be better than on Android because the PhoneGap API method calls may be sub-optimally implemented in the Android version. While PhoneGap’s writing to a file benchmarks are only marginally faster than native iOS, in the compass benchmark PhoneGap clearly outperforms both the native platforms and MoSync, Table 4.1. PhoneGap receives two and a half times

25 CHAPTER 5. DISCUSSION the number of updates compared to Android, and almost six times that of iOS. Since the native modules are implemented by PhoneGap themselves they could have included additional software logic to speed up the update frequency. They could, for example, be using cached values, that are returned to the caller if the compass has not received new updates within a certain amount of time. It could also be the case that they implement deprecated methods that return updates faster. Since the performance differs for each system resource, a prospective developer must carefully consider the impact of using the different resources that their application needs. Take for example a PhoneGap Android application, here the extra time it takes to write to a file might not affect the general functionality of the application, as much as the extra time it takes to access the geolocation resource. Even though it takes 191 times as long to write a 32 byte string to a file, Table 4.3, this only means an extra overhead of 15 ms. 15 ms does not affect the overall user experience unless I/O accesses are very frequent. On the other hand, the modest 10 times slower access time on the geolocation resource, which results in 12 updates per minute, is a significantly bigger problem. Imagine using a GPS while driving a car and only receiving updates every 5 instead of every 0.5 seconds. This translates to 139 meters between updates instead of only every 13 meters when traveling at 100 km/h. Aside from the inherent overhead some of these performance issues can be solved by implementing custom native plugins, where functionality and performance can be optimized for the developers purposes. They can be used to perform CPU heavy tasks where JavaScripts performance is just not good enough. They also allow developers to extend the native functionality of the PhoneGap API which might be necessary in the future as more features are added to the platforms all the time. This however, requires that the developer writes native code, something that we wished to avoid by using PhoneGap in the first place.

5.1.2 MoSync

Since MoSync recompiles the code into native byte code, instead of wrapping it in an abstraction layer, it should suffer less from overhead and thus have better performance than PhoneGap. This holds true for every benchmark except the compass where PhoneGap outperforms both Android and MoSync. MoSync performs slightly worse than native Android in the geolocation and compass benchmarks, Table 4.1. This is not too surprising since Android should realistically be better at optimizing their compiled code than MoSync. What may be surprising though is the fact that MoSync performs quite a bit better at all but one of the file system benchmarks, as can be seen in Table 4.2 and Table 4.3. This could be explained by the fact that Android supports compiling C++ code natively. According to Android [8] this seldom benefits the applications performance, except in certain circumstances. One of these circumstances are when performing CPU heavy tasks, and since file I/O is a very CPU heavy task it comes as no surprise that this is an area in which MoSync performs well. It should be noted that MoSync has not revealed exactly how the recompilation works, but it can be speculated that

26 5.2. COST EVALUATION it is using the native compiler whenever possible. A downside with MoSync is its natural rigidity. Where PhoneGaps framework is very flexible and can be modified and extended using custom native plugins, MoSync can only provide support for what they have already implemented. Since it recompiles the code instead of just wrapping it and providing a way to communicate with native code, the framework cannot be modified by the user. This becomes a problem when developers wants to solve a problem using functionality that MoSync does not currently support. MoSync and PhoneGap, as well as other companies developing CPTs, cannot compete with the resources of Apple and Google. They will seldom be able to stay up to date with all the new features. With PhoneGap this problem can be circumvented in some cases by implementing it yourself through plugins, while the very nature of MoSync as a source-code translator prohibits you from extending its functionality. An example of this is the inability to create new threads or processes in MoSync[6], a fundamental feature for many developers. PhoneGap does not support threading either due to the fact that JavaScript does not, however if it is required a custom plugin could be created that implements threads.

5.2 Cost evaluation

To start developing applications targeting both iOS and Android using native coding technologies, Tekis would have to start two project groups due to the differences between the two platforms. Following Tekis’ standard procedure when starting new projects this would necessitate the hiring of an expert for each of the platforms. If, however, Tekis would use a CPT to produce the same application they would only need one project group and one expert, which would effectively cut their costs in half. This is a very simplified argument but it serves to make the point that using CPTs is a cost effective way of developing applications. Since all the approaches use different programming languages and development environments they all require different skill sets. Thus the resources at the company’s disposal must be analyzed when evaluating what it would take for a company to start developing applications. As can be seen by the results from the questionnaire in Figure 4.1, Tekis AB primary development competence lies in web and Java programming. While five out of thirteen people answered that their skill with C and C++ is a 4 on a scale from 1 to 5, eight people answered 2 or less. The curve for both Java, JavaScript and HTML/CSS looks much better with a larger amount of answers in the range 3 to 5. The IDE which the company’s developers are most comfortable using is Visual Studio, with eleven out of thirteen answering 4 or more on how much experience they have with an IDE, Figure 4.2. Eclipse comes in second place with only four answering 4 or higher. Since PhoneGap uses the coding languages that Tekis’ developers know best and is compatible with the IDE that they are most comfortable with, this makes it

27 CHAPTER 5. DISCUSSION a more suitable choice than MoSync for cross-platform development in regards to development environment. This is because it would allow them to spend less money on education and less time on getting used to its development environment. This of course does not necessarily hold true for other companies. As stated above all companies have their own specialities and skill-sets, and thus need to perform their own analysis on which CPT best suits their own organisation.

28 Chapter 6

Conclusion

The choice on which approach to use, web-to-native wrappers or source code translators, based on a performance and cost analysis is difficult. It depends mostly on what skill-set and competences the target organisation contains, and what kind of applications they are aiming to produce. While source code translators like MoSync provide better performance, making them good for performance demanding applications such as games, the nature of how they function make them less flexible. Web-to-native wrappers, like PhoneGap, are much more flexible allowing developers more freedom in designing their applications. The price for this flexibility unfortunately, lie in their poor performance. The loss of flexibility when using source code translators is an issue. When choosing a CPT developers must be aware of which features that have not yet been implemented. Web-to-native wrappers with their modular approach, allow developers to fill their customers’ needs by themselves while source-code translators only allow the use of functionality that its developers have already implemented. The potential economic gains are great when using CPTs to target multiple platforms. How big these gains are depend on the techniques used by the CPT and skill set already existing within the company. The decision whether to use a source- code translator or web-to-native wrapper should only be made after considering the requirements of the application. If all the functionality requirements can be met by a source-code translator this is the better choice considering the performance gains. Generally however, web-to-native wrappers with their higher flexibility makes them a more stable and long term choice as a cross-platform development tool.

6.1 Future work

Cross platform development tools is an ever expanding subject. Understanding the complete performance implications of the diﬀerent approaches is beyond the scope of a thesis project. To improve upon this thesis however a more focused study should be performed. My recommendation is focusing on only one platform. This would allow for the inclusion of more CPTs or a larger selection of system resources

29 CHAPTER 6. CONCLUSION to be benchmarked. Including more CPTs in the study gives a broader spectrum of comparison while including a larger selection system resources allow for a deeper and more comprehensive comparison.

30 Chapter 7

References

[1] MoSync AB. Creating User Interfaces With MoSync. Visited in Jan 2014. url: http://www.mosync.com/docs/sdk/cpp/guides/ui/creating-user- interfaces-mosync/index.html. [2] MoSync AB. MoSync SDK. Visited in Jan 2014. url: http://www.mosync. com/sdk. [3] MoSync AB. The JavaScript NativeUI API. Visited in Jan 2014. url: http:// www.mosync.com/docs/sdk/js/guides/nativeui/jsnativeui-library/ index.html. [4] MoSync AB. The MoSync Toolchain. Visited in Jan 2014. url: http://www. mosync.com/docs/sdk/tools/guides/architecture/toolchain/index. html. [5] MoSync AB. The NativeUI C++ Library. Visited in Jan 2014. url: http: //www.mosync.com/docs/sdk/cpp/guides/nativeui/using-nativeui- library/index.html. [6] Mosync AB. Optimizing mobile applications. Visited in Feb 2014. url: http: //www.mosync.com/docs/sdk/cpp/guides/testing/optimizing-mobile- applications/index.html. [7] open handset alliance. Android. Visited in Feb 2014. url: %5C%5Chttp:// www.openhandsetalliance.com/android_overview.html. [8] Android. Android NDK. Visited in May 2014. url: https://developer. android.com/tools/sdk/ndk/index.html. [9] Android. Android, the world’s most popular mobile platform. Visited in Feb 2014. url: http://developer.android.com/about/index.html. [10] Android. Developer Tools. Visited in Feb 2014. url: http://developer. android.com/tools/index.html. [11] Apple. About the iOS Technologies. Visited in Feb 2014. url: https : / / developer . apple . com / library / ios / documentation / miscellaneous / conceptual/iphoneostechoverview/Introduction/Introduction.html.

31 CHAPTER 7. REFERENCES

[12] Apple. iOS 7 features. Visited in Feb 2014. url: https://developer.apple. com/ios7/#new. [13] Apple. iOS Developer Library. Visited in Feb 2014. url: https://developer. apple.com/library/ios/referencelibrary/GettingStarted/RoadMapiOS/ FirstTutorial.html#//apple_ref/doc/uid/TP40011343-CH3-SW1. [14] Apple. iOS Developer Program. Visited in Feb 2014. url: https://developer. apple.com/programs/ios/. [15] GSM Arena. HTC Desire. Visited in Feb 2014. url: http://www.gsmarena. com/htc_desire-3077.php. [16] A. Charland and B. LeRoux. “Mobile application development: Web vs. native”. In: Communications of the ACM 54.5 (2011), pp. 49–53. [17] Andreas Constantinou et al. Cross-platform developer tools 2012. Tech. rep. VisionMobile, 2012. [18] Luis Corral, Alberto Sillitti, and Giancarlo Succi. “Mobile Multiplatform De- velopment: An Experiment for Performance Analysis”. In: Procedia Computer Science 10 (2012), pp. 736–743. issn: 1877-0509. [19] Priori Data. Top apps and publishers report. Visited in Feb 2014. Jan. 2014. url: http://prioridata.com/products/free-reports/?top-apps-and- publishers-report. [20] David H Deans. Global Shifts in the Smartphone Platforms Market. Aug. 19, 2013. url: http : / / socialmediatoday . com / david - h - deans / 1677421 / global-shifts-smartphone-platforms-market. [21] D. Dern. “Writing small [Tools and toys]”. In: Spectrum, IEEE 47.6 (June 2010), pp. 14–15. issn: 0018-9235. [22] Philip J. Fleming and John J. Wallace. “How Not to Lie with Statistics: The Correct Way to Summarize Benchmark Results”. In: Commun. ACM 29.3 (Mar. 1986), pp. 218–221. issn: 0001-0782. [23] Rohit Ghatol and Yogesh Patel. Beginning PhoneGap: Mobile Web Framework for JavaScript. Apress, 2012. isbn: 978-1-4302-3903-1. [24] Mark H. Goadrich and Michael P. Rogers. “Smart Smartphone Development: IOS Versus Android”. In: Proceedings of the 42Nd ACM Technical Symposium on Computer Science Education. SIGCSE ’11. Dallas, TX, USA: ACM, 2011, pp. 607–612. isbn: 978-1-4503-0500-6. [25] Google. Application Fundamentals. Visited in Feb 2014. url: http : / / developer.android.com/guide/components/fundamentals.html. [26] Google. The Android Source Code. Visited in Feb 2014. url: http://source. android.com/source/index.html. [27] Adrian Holzer and Jan Ondrus. “Mobile application market: A developer’s perspective”. In: Telematics and Informatics 28.1 (2011), pp. 22–31. issn: 0736-5853.

32 [28] Mobile HTML5. HTML5 compatibility on mobile and tablet browsers with testing on real devices. Visited in Jan 2014. url: http://mobilehtml5.org/. [29] Global Web Index. Q3 GWI Launch: Mobile Operating Systems. Visited in Feb 2014. Oct. 2013. url: http://blog.globalwebindex.net/mobile- operating-systems. [30] jQuery Mobile. A Touch Optimized Web Framework. Visited in Jan 2014. url: http://jquerymobile.com/. [31] Masoud Nosrati, Ronak Karimi, and Hojat Allah Hasanvand. “Mobile Com- puting: Principles, Devices and Operating Systems”. In: World Applied Pro- gramming 2.7 (July 2012), pp. 399–408. issn: 2222-2510. [32] Arno Puder. “Cross-Compiling Android Applications to iOS and Windows Phone 7”. In: Mobile Networks and Applications 18.1 (2013), pp. 3–21. [33] Samsung. Galaxy Note 3 specs. Visited in Feb 2014. url: http : / / www . samsung.com/se/consumer/mobil/mobil/smartphones/SM-N9005ZKENEE- spec. [34] John M. Wargo. PhoneGap Essentials: Building Cross-Platform Mobile Apps. Addison-Wesley Professional, 2012. isbn: 0321814290. [35] Alexander Zibula and Tim A. Majchrzak. “Cross-Platform Development Us- ing HTML5, jQuery Mobile, and PhoneGap: Realizing a Smart Meter Ap- plication”. In: Lecture Notes in Business Information Processing 140 LNBIP (2013), pp. 16–33.