Development of Responsive Multi-Device Applications within Apple Ecosystem

Jingkai He

Supervisor: Dr Charaka Palansuriya

August 14, 2014

Contents

Acknowledgement ...... 1

Abbreviations ...... 2

Abstract ...... 3

1 Introduction and Objectives ...... 4

2 Background and Literature Review ...... 6

2.1 State of The Art of Multi-Device Application Development ...... 6

2.2 Background Knowledge of Apple Ecosystem ...... 7

2.2.1 Memory Management ...... 8

2.2.2 User interface ...... 8

2.2.3 Data Model ...... 8

2.3 HPC in Multi-Device Application Development ...... 8

2.3.1 Clang Compiler ...... 9

2.3.2 Grand Central Dispatch ...... 10

3 Functional Specification and Design ...... 11

3.1 Functional Specification ...... 11

3.1.1 Basic Functionality ...... 11

3.1.2 Advanced Functionality ...... 12

3.2 Architecture ...... 13

3.3 Design ...... 14

3.4 Library Design ...... 14

3.5 User Interface Design ...... 15

3.5.1 iOS Human Interface Guidelines ...... 15

3.5.2 Mac OS X Human Interface Guidelines ...... 16

3.5.3 User Interface for iOS Device ...... 17

3.5.4 User Interface for OS X Device ...... 17

3.5.5 Application Layout ...... 18

i 3.6 Model Layer Design ...... 19

4 Implementation ...... 20

4.1 Some Hardware and Software used in the Project ...... 20

4.2 Library Implementation ...... 20

4.2.1 Section and Subsection ...... 21

4.2.2 Paragraph ...... 22

4.2.3 Horizontal Line ...... 22

4.2.4 Rich Text among Paragraphs ...... 22

4.2.5 List ...... 24

4.3 Application Implementation ...... 24

4.3.1 Common Model Layer ...... 24

4.3.2 iOS Controller Layer Development ...... 25

4.3.3 OS X Controller Layer Development ...... 26

4.4 Design Pattern ...... 27

4.4.1 Delegation Pattern ...... 27

4.4.2 Active Record Pattern ...... 28

4.4.3 Monkey Patching ...... 28

4.4.4 Observer Pattern ...... 29

5 Performance Optimisation ...... 30

5.1 Profiling Approaches ...... 30

5.1.1 Hardware for Profiling ...... 31

6 Result and Evaluation ...... 32

6.1 Application Run on OS X device ...... 32

6.2 Application Run on iOS device ...... 33

6.3 Markdown Parser Optimisation ...... 35

6.3.1 Object Loading Optimisation ...... 35

6.3.2 Multithreading Optimisation ...... 40

6.3.3 Multithreading Overhead Reducing ...... 44

ii 6.4 Optimisation on OS X Platform ...... 48

6.4.1 Reduce Unnecessary View Rendering ...... 48

6.4.2 Asynchronous Non-blocking Rendering ...... 51

6.5 Optimisation on iOS Platform ...... 52

6.5.1 Multithreading Optimisation ...... 52

6.5.2 Table View Optimisation ...... 54

7 Conclusions and Future Work ...... 56

7.1 Future Work ...... 56

References ...... 58

iii

Acknowledgement

I would like to express my great appreciation to my supervisor, Dr Charaka Palansuriya, for his generous and thoughtful suggestions and guidance all through the dissertation.

I would also like to thank EPCC for providing me with great teaching resource and opportunity to work on the world-class computation hardware during the MSc study.

Finally, I would like to thank my parents, for their support and encouragement throughout my study.

1 Abbreviations

ARC Automatic Reference Counting

GCD Grand Central Dispatch

GUI Graphical User Interface

SDK Software Development Kit

UI User Interface

VM Virtual Machine

2

Abstract

Mobile devices, such as laptops, tablets and smartphones are increasingly being used in daily work. A successful application is expected to be available on many such devices. The project explores the state-of-the-art in developing multi-device applications with high responsiveness within Apple ecosystem.

To identify the methodology suitable for such multi-device development, a notebook application using Markdown syntax is developed on Apple devices. It is implemented to be compatible for devices with different screen size and take the advantage of device-specific hardware.

After the notebook application is developed, some high performance related technologies and tools are applied to optimise the performance and improve the responsiveness of the application. The techniques used include enable the application to run on multi-cores by using multithreading, reducing the CPU and memory usage by lazy loading objects and applying caching technology. The responsiveness of the application before and after the performance optimisation is then discussed.

3

1 Introduction and Objectives

Presently laptops, smart phones and tablets are one the most popular computing devices used by people. The markers of the operating systems running on those devices usually provide developer with Software Development Kits (to be called SDK for short) so that developers can build applications within the chosen software ecosystem. This can create a win-win situation: As developer can be profitable by making popular software, platform will also be popular due to quantity and quality of the software available on it.

A large number of successful applications nowadays are now built for different devices and running on different operating systems. One example is the Microsoft Office, which is built on Windows, OS X, Windows Phone, Android, iOS etc. On the other hand, building applications on multiple platforms is usually frustrating. Applications have to be optimised for devices with different hardware resources like number of cores available, screen size and resolution, etc. Due to the limitations of availability of programming language and SDK, applications sometimes also have to be programed in extreme different ways, which makes the development even harder.

An objective of this project is to investigate the state-of-the-art in developing multi-devices applications. The project explores some common methodologies to overcome the difficulties of multi-device application development. Currently multi-devices application development usually follows the concepts of ‘Mobile First’ or ‘Web First’1, if the development team is small. This means that the application is developed for one specific platform, such as Android first, and then for the others later. By using this approach the quality of the application for a single platform can be ensured, but usually takes a long time for the release on all targeted platforms. On the other hand if the development team is large, team sometimes is separated and is responsible for different devices, which saves time but might make project out-of-control if the sub-projects are not well managed. In this project, a methodology different from above will be applied in multi-device applications development. That is to develop applications for multiple devices simultaneously.

Significant number of phones, tablets and other mobile devices nowadays are equipped with at least dual-core processors. However most client side applications do not take the advantages of multi-cores but are running on single core or even single thread, since single-core can still achieve reasonable performance without process communication or thread switches. It is also easier to maintain than parallelism and concurrency software. On the other hand, to improve the user experience of the applications, most of applications have nice GUI and animations, come along with intense computation in the background. The inevitable trade-off is that lags and crashes exist within application. This definitely will reduce the responsiveness and usability of the applications. As an important objective, the project exploits the high performance features (for example, multithreading and memory optimisation)

4 available in the chosen platform. The aim is to create a highly responsive multi-device application utilising the high performance features.

Apple Ecosystem is chosen as the platform to be investigated. It is mainly due to its overall market share for both laptop and mobile devices2, as is shown in Figure 1. Moreover, Apple has provided developers with mature SDK: Cocoa touch framework for iOS development and Cocoa for OS X development. Last but not least, high performance library (For example, Grand Central Dispatch) to build highly responsive application is natively included within SDK. As a result, it is ideal to build a multi-device application within Apple Ecosystem.

Figure 1: Market Share of Mobile and Laptop Device.

5

2 Background and Literature Review

2.1 State of The Art of Multi-Device Application Development

Generally there are two approaches to develop multi-device applications. One is building native applications, which means to maintain several codebase for different platforms. The other is to build a-web view application, which is also known as hybrid application, runs inside a web-view container but performs like a native application.

Several articles are reviewed to explore the state-of-art of multi-device development. One of them is named Mobile Application Development: Web vs. Native3. In this article the author has introduced some of the basic good parts and weakness between native and web-view application.

In the author's point of view, web-view applications are much cheaper to develop and deploy than native apps, however the user experience, responsiveness and performance cannot catch up with native applications. Due to the nature of a web-view application, it can be simply built in HTML, CSS and JavaScript within a web-view container. The internal mechanics of the web-view container can then communicate with the underlying hardware so that the web-view can be responsible to situations such as drag-and-drop, word typing and photo taking etc. With the brand-new JavaScript Virtual Machine technologies, such as V8 by Google and SpiderMonkey by Mozilla, JavaScript code can be directly compiled and optimised to native machine code. The JIT (Just in Time) technology can even enable the JavaScript code being compiled in time.4 Those technologies make the performance of web-view applications much better than before, however are still not satisfied, as Facebook regarded their HTML5 based application their biggest failure. 5 Even though it is simpler to maintain web-view application, it means nothing to end-users if it does not have too much improvement to performance and user experience. Developing multiple devices in native supported languages, on the other hand, though takes longer time to develop, however worth the time considering the efficiency of execution and user experience.

After all, the approach of the development largely depends on the requirement and usage of the application. Since the responsiveness, performance and usability of the application is more in highly demand, native approach of development is then applied for multi-device application.

In general, every platform provides a programming language for development by default, such as Java for Android development, Objective-C for OS X and iOS development, C# for Windows and Windows Phone development etc. However for the purpose of make development simpler, some third party languages are also introduced, even though they are not natively supported. Xamarin, as a successful commercial cross-platform development framework, is a good example. On Xamarin platform, developers can write their application in C#. The code can be compiled to 6 Native iOS, Android, Window Phone, OS X and Windows application. Despite developers have to write platform-specific user interface, Xamarin can share most of the business logic and library, which makes development easier to some extent.

Xamarin sounds like a silver bullet for cross platform development, however comes with a series of trade-offs.6 The most serious issue is the application overhead. Volume of Xamarin applications is usually larger than average. This affects download time and disk storage on a portable device. It is mainly due to the reason that .NET code has to be statically compiled to platform specific assembly code, which is still not a mature technique. Another drawback is the limitation of UI that can be shared across platforms. As a result, developers still have to tweak platform-specific UI and event logics. The third issue is the ecosystem and community. In case Xamarin is developed by a solo company and is released as commercial software, the availability of community supporting and third party components is not as plenty as the natively supported one. Last but not least, the platform-specific optimization might be difficult in case an optimisation for one device might not benefit the others. Consequently though Xamarin can achieve both good performance and user experience, however might be a great tech debt due to the reasons listed above, therefore will not be considered as the choice of implementation.

As the application is built within Apple Ecosystem, how to enable the code to be shared between iOS and OS X operating system is also essential. An article named Creating a library to be shared between iOS and Mac OS X7 is reviewed, which introduced how to build modules written in Objective-C that can be shared across platforms. The article stated that an iOS application could only link static libraries when it is built. On the other hand, an OS X application can only be built with a framework. As a result, if a library is shared between two platforms, it has to be built into two targets: one for iOS and the other for OS X.

Another article is named Portability between iOS and OS X8, which introduces the main difficulties in porting applications between those two applications. It suggests that most of the user interface has to be redesigned during the cross platform development, however if MVC architecture is followed, a lot of code can be reused. Moreover, the ways of input of mobile and laptop devices are different. iOS devices mainly use touch and sensors, while OS X device mainly use keyboard and mouse input. When designing the UI, great efforts might be put on this.

2.2 Background Knowledge of Apple Ecosystem

As the multi-device application is built within the Apple ecosystem, it is also necessary to understand the main concept within it. After building several application examples and reading official documentation, it is noticed that most applications run on OS X and iOS uses Model-View-Controller 9 (to be called MVC for short) architecture, which is also one of the most common used approach to build applications on other platforms such as Android and Windows nowadays. To understand the state-of-the-art of multi-device development, it is crucial to understand the MVC architecture.

7 2.2.1 Memory Management Memory management is quite essential in multi-device development. As most of the mainstream smart phones only have 514MB memory, the memory has to be managed carefully to prevent memory leak.

In both iOS and OS X development, developers can not only manually manage the memory, but use automatic reference counting (To be called ARC for short) as well. Manual memory management can efficiently release memory that is no longer needed. Compare to garbage collection, manual memory management can prevent unnecessary memory accumulation, however also increase the complexity of development and maintenance, as developers have to take efforts to keep track of the life cycle of objects on low-level. Different from garbage collection, instead of monitor the code in the background, ARC technology enables compiler to inject code into the executable and then keep track of the reference counting of objects and release the memory of the certain objects that are not needed anymore10, which is regarded to be more efficient.

With the aid of ARC, developers do not need to work on memory management manually, which makes the memory management much easier, since it is generally regarded as the difficult part of C/C++ and Objective-C programming. The ARC technology is applied to the development of application, in case it can significantly reduce the budget of development and maintenance.

2.2.2 User interface The UI is also crucial, as it directly decides whether user will use the application or not. UIKit framework, as the fundamental SDK of iOS platform, provides developers with the infrastructure for creating graphic and event-driven user interface, by which developers can build applications on the iOS device. AppKit framework includes objects to build GUI and manage event loop for building OS X applications. Those two frameworks are significantly distinct between each other11, however share the same concepts of MVC design pattern. As a result, when developing OS X and iOS applications simultaneously, the code of graphic user interface might be significantly different.

2.2.3 Data Model The data model is provided as Core Data Framework by both OS X and iOS SDK. The data model is generally regarded as model layer in MVC architecture. If the model layer is crafted by following the proper design pattern, all device-specific program can share and reuse the data model without too many modifications.

2.3 HPC in Multi-Device Application Development

Applications across multiple devices are expected to be not only functional, but smooth and highly responsive as well. Nowadays, a lot of animations such as modal, pushing and flipping even some advanced transitions involve in the application to

8 ensure that the UI is interactive and looks vivid, which makes the application more understandable and useful. On the other hand, a consequence of using these is more CPU and memory usage, this in turn has an impact on the performance and battery life. The most direct impression left on the user is that the application suffers from lags and have a detrimental effect on the battery life. The situations might be even worse if the application have to perform some intense computation in the background, which might even block the user interface, if they are running on the same thread. Furthermore, if the CPU usage and memory usage accumulates to some extent, the operating system kernel of portable devices might simply kill the process of the certain application in the background, which is known as a crash. This delivers bad user experience if it happens quite often, which surely is not expected from developers.

To help improve the performance of applications, so that the computations resource consumed by UI animations can be reduced, manufacturers of operating systems on multi-device use different approaches. For example, Google, who is the maker of Android mobile operating system launched Google Project Butter that aims to make the operating system quicker by eliminating lags in the application, solving poor responsiveness and optimising love performance. On the other hand, those optimizations still cannot solve the issues caused by application developers.

To ensure that the application built in this project is performing well, responsive and without lag, some HPC technologies will be applied into cross-platform application development. In the next few sections, the high performance features in Apple ecosystem that can be used to improve the performance of application will be discussed.

2.3.1 Clang Compiler Clang is used during the development, in case it is the only compiler natively supported by Apple devices. Clang is a front-end compiler for the C, C++ and objective-C programming language. It uses Low Level Virtual Machine (LLVM) technology to support its back-end. It is generally regarded as a replacement of GCC compiler.12

Compared to Clang, GCC has several pros. GCC compiler supports languages that Clang does not support, such as Java, Fortran, go, etc. Some language features of GCC are not supported by Clang, such as nested function. On the other hand, Clang still has great advantages over GCC. The most essential one is that Clang is much faster and uses far less memory13. By using the technology of Automatic Reference Counting (ARC), string referenced objects will be released automatically by the compiler when those objects are no longer used by the program. It significantly reduces the budget of maintenance. The Clang compiler itself also provides extensions for ensuring thread safety, which will bring great convenience when doing multi-threading programming. Moreover, compare with GCC, which is not well documented, Clang follows the concepts of API design. The AST and back traces it provided are more comprehensive than GCC. When doing the profiling, great advantage can be taken by using Clang, as the CPU and memory usage, energy

9 impact, even multi-core load of each instrument can be traced by the profiling tools it provided, which significantly helps to find the bottleneck of performance.

2.3.2 Grand Central Dispatch To achieve great performance on multiple devices, multitasking is introduced, as it can efficiently schedule the computation intense work within the application. By using it, UI animations can be asynchronous with intense computation. Due to the limitation of the processor, the performance cannot be compared with the super-computer. The multitasking within application cannot achieve good parallelism, however benefits from the high concurrency can still enable the application to be more highly responsive.

Grand Central Dispatch (to be called GCD for short) is a technology applied in Apple devices with multi-core processor(s) to support multi-core programming. 14 It is introduced since the release of OS X 10.6 and iOS 4, mainly implemented in thread pool pattern, which is fundamentally a queue mechanism. It generally maintains several thread-safe queues, with different priorities, working in “First in, first out” (to be called FIFO for short) order, however it does not guarantee the completion of work. The work out of the queue is run in a thread pool maintained by GCD. The load-balance among threads is done by the OS kernel, consequently the amount of parallelism and concurrency highly depend on the current load of CPU and memory, as the OS kernel have to switch frequently among processors to schedule different applications.

Compared to traditional Posix thread model, it is simpler to do multithread programming by using GCD, as most of the load balancing among multiple cores and switches among threads are maintained by GCD automatically. 15 Moreover, the performance is not worse than traditional thread programming. It is supported in both OS X and iOS operating systems at application level. Consequently it is applied as a way to improve the performance.

10

3 Functional Specification and Design

To implement the methodology of developing applications for different device-platforms simultaneously, a responsive application for multiple devices is then developed. The application chosen to be developed is a notebook. The reason for choosing to develop a notebook is because by doing this, it is easier investigate as much components as possible, in case a notebook application contains plenty of the functionality that is also available in other applications. To follow the methodology, the notebook application is developed for both iOS and OS X platforms simultaneously. By this way, applications run on different devices can have same or similar functionality so that comparisons can be made among them. Moreover the program can be designed to be compatible for both two operating systems as much as possible. Last but not least, the difference of memory usage and computation intense parts among multiple devices can be noticed by working on the same or similar functionality, which can help to understand the methodologies and state-of-art of developing high performance and responsive application across platforms. In the next few sections, the functional specifications are discussed to decide the basic functionality. At last, the UI is designed and created based on the functional specification, bases on which concrete implementation can be applied.

3.1 Functional Specification

The functionality of a notebook application is relatively simple, as the main duty of it is simply to make note keeping easier. As simply plain text editing obviously cannot meet the need, the notebook application is developed as rich-text editing software run on multiple devices. Moreover, as most of portable devices provide plenty of hardware components such as camera, keyboard and microphone, advantages can be taken by interacting with the underlying hardware components, which is regarded as the advanced functionality.

3.1.1 Basic Functionality As a notebook application, besides plain text editing functionality, which is obviously required, some rich text-formatting feature is also a must. Like some other text editing software, such as Microsoft Word and Evernote, the notebook application should support some basic formatting that help separating the content logically, such as titles, sections, emphasis, bold and so on. The difference is that the aim of the project is to help the user such as scientists, researchers, engineers and developers. As the nature of their works, they trend to doing note keeping fast and well-organised, instead of adding nice decorations via a heavy menu driven GUI. As a result, the notebook will be developed in lightweight, which can provide the users with the most basic formatting functions so that they can take notes in a fast and easy way. In the following paragraphs, the basic functions will be discussed in details.

11 The most basic formatting feature that the application provides is the paragraph, which is used to express the meaning with an aggregation of several sentences.

Title, section and subsection will also be supported in the notebook application. Instead of the heavyweight formatting in Microsoft Word, the titles, sections and subsections in the app are relatively simple, simply distinguished by six different headers with different size. By this the note keeper and reader can still easily tell the level of the section or subsection.

During the note keeping, it is quite usual for user to highlight on something important, so some formatting that support emphasis will also be supported, which includes emphasis, as known as italics, strong emphasis, as known as bold and strikethrough, which is used to quickly delete some words.

Links and references that attach the resources outside the note are also supported. Those include link, reference, email address, website URL, videos and images. In the design section, the details about how those features can be added will be discussed.

To keep something that should be displayed in order, people usually use lists and tables. The notebook supports those two formats. In the list and table of the application, user can still do rich text editing such as adding link and doing emphasis.

When the researchers and engineers are doing note keeping in a hurry, it is quite usual for them to add some contents that are not quite related in the same note. In order to separate the unrelated contents, horizontal rule is introduced so that a break line can be added between two of them.

3.1.2 Advanced Functionality When keeping notes, it is important that the user is able to quickly attach pictures by adding them from existing file or directly taking by camera by devices. The attachment adding functionality should be implemented to the application, as pictures and videos are the most direct way to explain the progress and work nowadays.

With the increase of the number of notes taken in the application, searching functionality should also be provided so that the content can be filtered. As a result, user can then find some specific notes much easier.

As the content of the notebook application can be previewed, the preview content should also be able to export out of the device, so that the edited note can be shared, even on other devices via email, short message and social network etc. As so, an “export as PDF” functionality will also be implemented.

When a user takes a note on the mobile phone, it is also quite common for him or her to expect continuing editing the note on other devices, such as a tablet or a desktop. As a result, data synchronisation between iOS and OS X devices should also be implemented. It is noticeable that most commercial cross platform products nowadays support data synchronisation via cloud platform. For the notebook application, data of notes should be synchronised by iCloud or a secure remote server.

12 3.2 Architecture

Like most of the client side applications, the development of the notebook application follows the MVC architecture for both iOS and OS X. By using this pattern, developer can easily separate the internal business logic from how the information is presented to the user. As the name of MVC tells, the pattern is mainly consisted by three layers: model, view and controller. In the following sections, the main concepts of those three components and how they are concretely implemented is discussed so that clear idea about how the application is architecture can be confirmed.

The model layer encapsulates the data specific to the application and the business logic about how to manipulate on the data. In a common client side application like the notebook, business logic is usually not quite heavy. Most of the work in the model layer is about data persistence and how the data of the application is organised. Since the data of the notebook is expected to have persistency, database will be introduced to store the data. Core Data library provided by Cocoa library will be used to mapping the database to Objective-C object.

The view layer is of MVC pattern is also known as the user interface, which generates the output and represents it to the user. It can be generally regarded as what users see on screen. The development of the view layer can be generally regarded as graphic user interface design and implementation. The Storyboard and Xcode Interface Builder will be used to craft the user interface.

The controller layer is the connection between the model layer and view layer, which deals with the event requests triggered by users, data updating, view displaying, animation, view layer’s delegation and so on.

Figure 2: The MVC architecture of Cocoa Framework

Figure 2 shows the main workflow and collaboration by using MVC pattern. In general, when using the MVC architecture, the whole interaction between user and the device can be described as a life cycle. Initially the user views the UI, he or she can trigger a user action by operating on the view layer. After that, the action will be sent to the controller. The controller will then send the message to the model. When the model receives the message sent by controller, it will reorganise and update the data it mapped and then send notification to the controller along with resources that is needed by controller. The controller at last updates the user interface as what it is 13 programmed to do. Bunches of those life cycles consists the interactions between user and application.

3.3 Design

Previous section describes the overall architecture of the application. As how the application should be organised is confirmed, the main focus then is put on the design of the application, which is consisted of library design, user interface design, model design and the connections among them. Following section discusses about the concrete design decisions.

3.4 Library Design

Keeping notes on a rich text editor is usually regarded as complex. One aim of the project is also to make it simplified. A common text editing application is usually a rich text editor, with massive of buttons, toolbars and options. A user might never use a large amount of them. To some extent too many options makes it hard for user to make a choice. To make keeping notes simple and ensure that most of the functions is useful, markdown format is introduced into the project. It is set as the only text-editing format for the multi-device application.

Markdown is a markup language that converts plain text into HTML. It is widely used in readme file of software source code, online documentation and discussion. The Markdown syntax is created by John Gruber and Aaron Swartz16, with the goal of allowing people to write readable plain text that can be easily viewed in web browser and displayed online. There is no community standard for Markdown syntax. To make it clear, Github Flavoured Markdown17 syntax is introduced as the standard of the project, with some specific extensions such as photo taking and uploading feature that can make the client side note keeping simple.

As Markdown syntax is not implemented in the SDK, a Markdown parser has to be developed to translate the plain text in Markdown syntax to HTML format. In case the whole project is planned to be written within Apple ecosystem, a markdown is written in Objective-C. Regarding the fact that there are varies of Markdown library writing in JavaScript, those library can be simply imported into web-view container, however it is almost impossible to optimise the performance within the web-view container, as web-view container is run upon a WebKit virtual machine, which is regarded as a black box. Moreover as JavaScript has GIL (Global Interpreter Lock), it is impossible to improve the performance of the Markdown library as the threads of which cannot be run on parallelism. On the other hand, when the code is written in Objective-C, optimisation can then be done lately via introducing the GCD technology to enable the Markdown parser run on parallelism and concurrency.

The Markdown syntax, as the way of note keeping within the application, is the main functionality of this project. It is developed first with minimal implementation, after which it can be then integrated into the notebook application. By using the multi-device application, real bottleneck or flaws existed in the library will then

14 reveal and be improved. Both functional application and stable library can be built through this approach.

The Markdown parser library is named MicroDown, as it is a minimal implemented. The code will be written in object-oriented principle provided by the feature of Objective-C to ensure that the code base is well organised and easy to maintain and optimise.

Originally, traditional procedures of developing a compiler are consisted of lexical analysis, parsing and code generation, however regarding the fact that the Markdown syntax is hard to do lexical analysis, it is then analysed by using regular expression. In details, the general patterns of markdown syntax will be listed. When a text fragment matches one or more patterns in regular expression, it will be converted to the certain data structure and inserted in the parse tree. The analysis and parsing work will be done simultaneously due to the nature of markdown syntax. Compared to the traditional compiling methodology, it is expected to be more efficient. Once the tokenisation and parsing work is done, the token tree would be translated into HTML format. Since it will always be a format matching a certain pattern listed, the translation work is regarded to be simple.

3.5 User Interface Design

For OS X and iOS applications, the user interface is also known as the graphical user interface. Though both of the two operating system use XNU as their kernel, the look and feel of the user interface is quite different, as one OS is for mobile while the other is for laptop. Due to this fact, the principle of the UI design is also different.

Before designing the user interface from scratch, guidelines about the user interface design principle should be studied. Fortunately, Apple provides Human Interface Guidelines for both iOS and OS X platform.

3.5.1 iOS Human Interface Guidelines According to the iOS Human Interface Guidelines, iOS 7 is consisted by 3 core philosophies: deference, clarity and depth. 18

Deference means that the content and the core functionality is always the most important part of user interface design. Everything else comes the second. When the user is using the application, the most important content should always be noticeable and the theme should always be simplified (Unless the application is a game or some other specific software in which heavy textures or 3D effects is a must to it). The necessity of a less important function should always be doubted. If the less important function is developed, it should always be put into the less important place. The guideline suggested that the core functionality should always be developed first, which is the concrete implementation of deference.

Clarity means to make things obvious. Everything displayed on the screen should be unambiguous. In concrete, text should be made readable; Icons should be designed

15 identifiable; Content should clearly indicate what it does. Last but not least, color is also meaningful. For example, red means destruction; green means approval and blue means progress. Buttons or links with improper color can easily confuse the user. Red, green and blue color should wisely indicate the usage.

Depth is an abstract concept, which everything displayed on the screen should be transitional. In real life, when a person walks from a room to another, he can feels the changing of space and distance or he will feel at lost. The same concept can be applied to user interface design. Animations are heavily used in iOS programming as an infrastructure library to makes the user feel the depth of the application.

3.5.2 Mac OS X Human Interface Guidelines OS X is a desktop operating system, as more powerful than iOS operating system it is, the user interface of it is also more complicated. The GUI theme of OS X is named Aqua. As its name suggests, the look and feel of the GUI is water-like. According to the OS X Human Interface Guidelines, OS X is driven by 4 core principles: familiarity, simplicity, availability and discoverability.19

Familiarity suggests that the user interface of OS X should be a reflection of the real life. In details, that is to use icon and text to explain the button and gather the components with similar or contrary functionality together and provide it to user, which ensure that the application is user friendly.

Simplicity means to focus on the main functionality of the application. The design should serve the functionality. The principle of simplicity is similar to the deference philosophy of iOS, as they share the design concept to some extent.

The concept of availability is similar to the simplicity principle to some extent. It suggests that all components, even not important, its feature and functionality should not be hide too deep to the user so that user can gain it without too many difficulties.

Discoverability suggests that the application should provide cues about how to use the functionality and if it is available. In details, if a function is disabled, the button that can trigger the certain function should be indicated as disabled so that the user will not try to click the button. On the other hand, if a function is enabled, user interface should also have strong evidence, which indicates the availability of the function.

As some best practice of user interface design is discovered, the user interface design can be started. For those two different operating systems, different methods are used on crafting the UI. The traditional XIB technology is applied to craft the user interface of OS X application in case it is the only approach. For iOS application, storyboard technology, which is introduced in Xcode, is used to create the user interface. It is regarded to be easier but complex. Unlike the server side application development, the UI is developed first, simply because it helps to discover the functionality that is common for both two platform and notice the platform specific functionality. Moreover the main difficulties of compatibility between different operating system and different size of screens can be noticed from crafting several different UI. By developing the user interface first, screen compatibility issues can also be solved.

16 3.5.3 User Interface for iOS Device For most user interface design in industry, the designer has to create pixel perfect prototype first and then asks developer to implement the features within prototype, which makes the life cycle of application development even longer. Thanks to the user interface builder provided by Xcode 5, UI prototyping and implementation can be worked on the same time in it. Nothing prototyped in the user interface builder will be wasted as the user interface can be directly built and run as a real product, which also brings great confident to developer at the beginning stages. It is by far considered as a more convenient tool to build the user interface than other heavyweight tools like Photoshop.

The user has to view and edit the content of a note, as well as the overall information, such as a table of content. The iOS application will be built as a table-based application. When a user launches the application, he or she will see a table view with a list of titles of the notes kept. When a note is selected or created, the application will then create another view that shows the editing field of the note. User can then keep note on the text field. The screen size of iOS devices has great impact on the design. In case the size of screen is quite limited, the view for editing and the view for preview have to be split into two different views, which means that after the user have edited the plain text in markdown format, by clicking the ‘Preview’ button, the preview page, which is rendered in HTML format, will be displayed. The scratching of the user interface mainly follows the thought described above. At last, the prototyped user interface is crafted in the Storyboard as is shown in Figure 3.

Figure 3: The User Interface and Workflow of The Note Application for iOS

3.5.4 User Interface for OS X Device As is mentioned in Mac OS X Human Interface Guidelines, unlike iOS that developer can build the user interface by Storyboard, only Interface Builder can be used to craft the user interface for OS X platform. Like the Storyboard for iOS, OS X application developer can also design and build the user interface simultaneously. Regarding the fact that the screen size of an OS X device is always larger than the size of any of the iOS devices, as smallest screen size of OS X device (Macbook Air) is 11 inch,

17 however the largest screen size of iOS device iPad is 9.7 inch. Different kinds of layout have to be applied to OS X platform. In case the screen size is larger, the application is designed as a single view application. The single view layer of the application is consisted by three split sub-views. The left-most side is the table of content, which log all the notes that is created and edited. User can trigger actions of creating, deleting and editing on it. In the middle is the editing view, on which user can editing the content of the note and save it. The right-most side is the preview of the note. The note is designed be real-time rendered, which means that every time the content of the current note is changed, the preview content will also be changed. By design the view like described above, the advantages of big screen can be taken to make the application on desktop platform more functional. By developing the application in a single page, it is not only more comprehensive, but enable user doing quick note keeping as well. The final design of the user interface for OS X platform is shown as Figure 4.

Figure 4: The User Interface of the Note Application for OS X

3.5.5 Application Layout To ensure that the UI is compatible with screens of different size, auto-layout technology is introduced into this project. By using auto-layout, UI components are not set with fixed size, but fluid and related, so that the size of them will be changed with the changing of the screen size. For the OS X application, it is quite common for user to enlarge or shrink the size of the application window, the size and position of the UI components have to be changed with the change of the size of application window, as is shown in Figure 5. For iOS applications, auto-layout are set to ensure that it the layout will be suitable for iOS devices with different size of screens, such as iPhone 4s, iPad, iPad mini etc., as is shown in Figure 6.

18

Figure 5: Note Application on OS X with different screen size.

Figure 6: Different Layouts on iPhone and iPad via Auto-Layout.

3.6 Model Layer Design

So far, as the user interface is crafted, main emphasis will be put on the development of functions. Since the MVC pattern is followed, business logic, display and control will be separated developed. As to the model layer, there will be only one model. That is the note, which has two keywords: title and content. To ensure that the data of the model can be persistent, the data will be stored into the database by using the Core Data component introduced by apple. As the Core Data component is cross iOS and OS X two platforms. The code of data storage and model layer on two platforms is expected to be the same.

In the MVC pattern, controller layer is regarded as the bridge between model layer and view layer. Since the user interfaces of the two platforms have great differences, the controller layer is also expected to be different.

19

4 Implementation

In the previous sections, the library, user interface and layout of multi-device application are discussed and have an overall idea about how the application should be designed. Main focus is then put on the implementation of the Markdown parser and the application. The application is implemented by the methodology stated in Introduction and Objectives, which is to develop the application on iOS and OS X platforms simultaneously. For example, when a feature of the notebook is implemented on iOS, it will be immediately migrated onto OS X platform, or will be abandoned if not suitable. This is expected to be not only efficient, but can enable applications on multi-device share as much functionality as possible. After the implementation, it is proved to be correct. In the following subsections, the main procedures of the implementation are introduced.

4.1 Some Hardware and Software used in the Project

In case development of iOS and OS X applications can only be worked on OS X system, so the main development work is done on a MacBook Air machine, on which OS X application and iOS simulator can run smoothly. To make sure the user interfaces of the iOS application work on correctly, iPhone and iPad devices are also introduced to be run on.

Xcode is chosen as the only development environment, as it contains almost the whole tool chain for OS X and iOS application development. To keep track the schedule of the development, revision control are also introduced. Git is used to do revision control. The project is hosted on Github, which is a web-based hosting service, which offered Git revision control. The main goodness of hosting the code online is to keep a complete backup of revision control. As the source code is distributed, when the data on the local machine is stolen or lost, the remote data can be pulled to the local so that the development can be continued.

4.2 Library Implementation

Regarding the fact that the core functionality of this notebook application is to convert plain text in Markdown syntax to comprehensible HTML content, the Markdown parser is the most important implementation for this application. The functionality is extracted as a stand-alone library for the reason of maintainability, as both OS X and iOS notebook applications have to use it.

As a parser the library is, the lexical analysis should be implemented first. As is discussed in Library Design, the library follows Github Flavoured Markdown as the guide of implementation. As a minimal implementation, some of the Markdown syntax such as code and syntax highlighting is not developed, as these features are not used in quick note keeping of daily life. In the following subsections, the main

20 procedures of the markdown parser implementation is introduced and discussed in details.

4.2.1 Section and Subsection In HTML syntax, no rules restrictedly define section and subsection. Actually section and subsections in HTML is called header. Generally the headers are consisted by h1, h2, h3, h4, h5, h6. The smaller the number is, the large the header is. In the Markdown syntax, the numbers are substituted by hash tags. The rules can be described as is shown in Table 1.

Markdown Syntax HTML Syntax

# Hello world

Hello World

## Hello world

Hello World

### Hello world

Hello World

#### Hello world

Hello World

##### Hello world

Hello World

###### Hello world

Hello World

Hello world (With blank line(s) above)

Hello World

=== (More than three equals)

Hello world (With blank line(s) above)

Hello World

--- (More than three dashes)

Table 1: Rules of Heading

Since analysing the syntax by typical lexical analysis is complex, regular expression is introduced to recognise the heading patterns existing in the Markdown format plain text. Fortunately, the regular expression modules are included in the foundational library for both iOS and OS X SDK so that they can be imported and invoked directly. The heading pattern consisted by numbers of hash tag can be identified as Equation 1 and the heading pattern consisted by equals or dashes can be identified as Equation 2. With patterns of the heading concluded, the existences in the plain text can be found and convert to the certain heading without too many difficulties.

A(#{1,6}) ([^\n]+?) *#* *(?:\n+|$

Equation 1: Pattern of Heading with Hash Tag

"\\A={3,}\\s* Equation 2: Pattern of Heading with Equals or Dashes under.

21 4.2.2 Paragraph A paragraph is a set of sentences that used to express a point of idea in writing. It is regarded as one of the most important elements in note keeping. In formal writing, the patterns of a paragraph is usually recognised by blank line. Sets of words split by several blank lines can be regarded as paragraphs, even if there is some line-break in the certain set of words. During the development, since the pattern is easy to be recognised, regular expression is even not introduce to implement as long as the pattern of the blank line is confirmed as is shown in Equation 3.

\\A\\s*$

Equation 3: Pattern of Sets of Blank Lines.

4.2.3 Horizontal Line When doing quick note keeping, it is quite simple to separate two unrelated content by using horizontal line. Consequently the Markdown parser also implements that feature. According to the guideline of Github Flavoured Markdown, every time a user type equal and dash symbols more than three times and the previous line is a blank or nil, the current line would be rendered as a horizontal line. The pattern of the horizontal line is also quite simple, as is shown in Equation 4. With the pattern discovered, the horizontal line lexical analysis can be implemented easily.

\\A[\\*-=]{3,}\\s*

Equation 4: Pattern of Horizontal Line.

4.2.4 Rich Text among Paragraphs As rich text rendering is the key feature of Markdown syntax, some useful syntax of Github Flavoured Markdown is implemented and modified. Moreover, some syntax that is included in Github Flavoured Markdown is put into practice. That includes word strong and emphasis, references, email link, image link, video link as well as YouTube link. The example, pattern and HTML format is shown in Table 2. As is shown, the height and width of image and video can be resized, which makes the rich text view more flexible.

22

Name Example Pattern HTML Format

jaxihe@gmail m> A-Za-z]{2,4})>

<((http|https|ftp): Auto //((\\w)*|([0-9]*)| ([-|_])*)+([\\.|/]( target="_top">jaxihjk.me |_])*))+)>

Your Website .+?)\\) Link (Blog website)

\\*([^\n\\ Bold *Bold text* ][^\n]+?[^\n\\ Bold text ])\\*

\\*\\*([^\n\\ Italic **Italic text** ][^\n]+?[^\n\\ Italic Text ])\\*\\*

~~Text is ~~([^\n\\ Delete Text is deleted deleted~~ ][^\n]+?[^\n\\ ])~~

[Image \\!\\[(.+?)\\]\\s*\ =100x100) 1}\\)

Table 2: Patterns and Formats of Rich-Text

23

4.2.5 List To keep note, listing something in order is quite common. Consequently, the Markdown Parser has provided list functions, which include ordered list and unordered list format. By using list, the note can be more well organised. To use the ordered list, what a user needs to do is simply typing the number before the content as is shown in Equation 5. As to use the ordered list, change the number with star or dash symbol, which is shown as Equation 6. The concrete pattern of the list is shown as the regular expression below. As the regular expression works on well, the list content in the plain text in Markdown format can be converted and rendered to HTML.

"\\A[\\d+]\\.\\s+(.+)

Equation 5: Ordered List Pattern

\\A[\\*-]\\s+(.+)

Equation 6: Unordered List Pattern

As is discussed in the design section of library, compared to the traditional compiler procedures, lexical analysis and parsing works will not done separately. The Markdown Parser pre-process the tokens in the lexical analysis procedure, so that it does the parsing work at the same time. In other words, after the lexical analysis is done, the grammar tree of the Markdown format text is also generated. After that, the grammar tree can be then generated to HTML text by using the pre-order traversal. By then, the fundamental functionality of the Markdown Parser is implemented. Emphasis can be then put on the implementation of the backend of the cross-platform application.

4.3 Application Implementation

The implementation of the applications is consisted of three parts: the model layer shared cross the note application on iOS and OS X platforms, the controller layer of the iOS note application, and the controller layer of the OS X note application. In case the user interface is scratched during the user interface design by using Xcode 5, the implementation of the controller layer can simply follow the user interface. In the next sections, the development of the model layer and controller layer will be discussed. As the fact the user interface of those two applications are distinct mainly due to the reason that they are on different device platforms, the backend implementation is discussed separately. Somehow some code base can still be shared in case the two note applications are developed simultaneously.

4.3.1 Common Model Layer As is introduced in the design section, in MVC architecture, model layer stands for the business logic and storage strategies of the application. The model layer of the multi-device application is consisted by only a note model, which has two attributes:

24 title and content. Consequently model layer can be shared across the application. In practice, the model layer is simply crafted for the iOS application and then the code is copied into the OS X application with some important minimal configuration and modifications, as both of the model layer is based on the Core Data library provided by Apple. For simplicity, only the implementation of the model layer of the iOS application is introduced, as the implementation of OS X version is basically the same.

For the data persistency, the data of the notebook is stored into the SQLite database, which is a lightweight file-based relational database. After that, the core data object can map the Objective C class to the database by Core Data adaptor directly. Thanks to the user interface provided by Xcode, the database and note table, which is consisted by title column and content column, which can be added by creating xcdatamodeld file and adding the table details to the configuration file. The details of the note table are shown in Table 3.

Attribute Type

content String

title String

Table 3: Columns of The Note Table.

After the database content is created, the Core Data model of the note table can be created. The note data model inherited from the NSManagedObject. Developer only needs to add getter and setter method of the content and title attributes to the Note class. The database adaptor of the Core Data library would automatically link the class to the data from the table of the database. As Objective-C 2.0 has provided the dynamic property feature, the code snippet is simply as shown in Code Snippet 1.

@interface Note : NSManagedObject @property (nonatomic, retain) NSString * content; @property (nonatomic, retain) NSString * title; @end

@implementation Note @dynamic content; @dynamic title; @end Code Snippet 1: Header and Implementation of Data Model of Note

4.3.2 iOS Controller Layer Development The controller layer of the iOS note application is mainly based on the design of the user interface on Storyboard, as set, is consisted by three controllers: NotesTableViewController, NoteViewController and MarkdownPreviewController. The functionality of those three controllers is as their name described.

25 NotesTableViewController is used to control and organise the list of notes that is stored into the database. User can trigger adding, editing and deleting actions on the table view and the message will be sent to the NotesTableViewController to deal with. To operate on the table view components provided by iOS software development toolkit, delegation pattern is applied to implement this feature. The details are discussed in the Delegation Pattern section.

Every time the user triggers the action of creating and editing to the NotesTableViewController, the backend will then display the content that NoteViewController rendered. The main duty of this controller is to handle the note editing actions, which include content changing, photo adding and taking, getting the title of content, as well as most of the UI animations. Due to the screen size limitation, some animations of the screen will force other components on the screen changing. For example, when the keyboard pops up from the bottom of the screen, the text area above it has to scroll up, or it will be covered by the keyboard box. To implement this, the text area scroll up event will be serialised as a notification and pushed into a notification queue on another thread. When the controller, as the delegator of keyboard received the message, it will then change the size of the text area and scroll it up automatically. Detail of the implementation is discussed in the Observer Pattern section.

After the user finish editing the note, he or she can then press the ‘Preview’ button. The action will be send to the NoteViewController, it will invoke the Markdown parser, which is implemented as a common library. The Markdown Parser will convert the plain text in Markdown format into HTML format. The title of the note will also be confirmed by parsing the first line of text. After that, the title and content of the note will be sent to the MarkdownPreviewController, which will sent the HTML content as plain text into the web view container embedded in it. The web view container will call the WebKit Virtual Machine and render the plain text into the final HTML content.

As iOS is the operating system for mobile device, some platform specific feature are provided for the application. To fully use the hardware, photo-taking functionality is integrated to the note application. User can take a picture from the iOS device and save it to the notebook. To implement this, image picker object is delegated to the NoteViewController.

4.3.3 OS X Controller Layer Development The controller layer of OS X note application is based on the previous designed user interface. In case the prototype has only one view, the application will have only one controller, which is named as NoteController. As the actions can be triggered from the view is almost the same number for iOS application, however built in only one controller, the NoteController is more complex. It provides creating, editing and creating action on the notes as the basic functionality of the note application.

In case the desktop machine provides physics keyboard, some short cut are also provided to trigger the actions on the notes, which brings a lot convenient to quick

26 note keeping on desktop. As the fact that UIKit that build iOS application is based on the Appkit, which is used to build OS X application, even the behaviour of the user interface is quite different, a large amount of code can be shared on the controller layer. For example, the image picking functionality and table view delegation is quite similar. When the image picking function is built on iOS platform, the function can be migrated to OS X platform without too many modifications.

In the design sections, the OS X note application is designed to have real time rendering feature, which means that every time the content of the note is edited, the preview area will be rendered again to show the changes. To implement that, the text view’s delegator is set as the NoteController and textViewDidChangeSection is implemented in the controller to observing the change of content. This brings great user experience, however have large CPU and memory usage. Further performance optimisation is introduced and discussed in Performance Optimisation section.

4.4 Design Pattern

To enable the applications being stable and well organised on multiple platforms, some design patterns are applied to development. Besides the MVC pattern used as the default architecture, some other design patterns are heavily used in the development in order to ensure that the code is maintainable and understandable. In the following sections they are introduced and discussed.

4.4.1 Delegation Pattern One of the most heavily used patterns is the delegation pattern. The delegation pattern is a design pattern in object-oriented program where an object delegates the task to a related object instead performing the tasks itself. On both iOS and OS X platform, the delegating object is usually an object generated by the library and the delegate object is typically a custom object created by the developer. One of the most common examples of the delegation pattern in Cocoa framework is the NSWindow class. NSWindow class declares a protocol with a method named windowShouldClose. When a user clicks the close button in the window, the window object sends windowShouldClose methods to its delegate to ask the confirmation of whether to close the window. The delegate will then return a Boolean value that controls the later behaviour of the window.

In both of the notebook applications on iOS and OS X platform, the table view object delegates its tasks to the notes' controller so that the object of notes' controller can carry on the tasks of the table view's object.

Another issue solved by using delegation design pattern is about the keyboard dismiss issue on iOS platform. When the text editing on the text view is finished, by clicking the area outside the text view, the keyboard box would not dismiss as expected, which does not provide user with good user experience. Without using the delegation pattern in objected-oriented programming, this function is quite difficult to implement. Fortunately, UITextView class provide a set of protocols that can delegate to other object. By delegating - (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent 27 *)event method to the NoteViewController object, and override it, the keyboard dismissing can be triggered by click the area outside the text box.

The image loading and taking feature is also a use case of the delegation pattern. The protocols of image picking and photo taking is delegated to the controller object so that we can load the image library and camera from the controller itself, which reduced the complexity of programming. Without this powerful feature, the most straightforward way is to create a new controller to deal with those operations, which will sure raise the complexity of the project.

By using the powerful feature of Objective-C, most of the tasks provided by UI components can be delegate to the controller object, which reduce the complexity of the program greatly and make the program more readable and understandable.

4.4.2 Active Record Pattern Another pattern used by the project is the active record pattern. In software engineering, active record pattern is an approach that accessing the underlying data from relational database by mapping it to the objects in object-oriented programming language. Developer and related user can directly do operations such as insert, delete and update and read on the objects which mapping the data, and the set of data will be updated automatically. When developers operate on the underlying database directly, they have to query the database by using SQL language, which is a potential issue that can create bugs since SQL language is regarded complex. By using the active record pattern, developers can operate on the objects in object-oriented language instead, which is regarded easy approach.

In iOS and OS X application programming, Core Data library is provided by Apple, which maps the data from database to Objective-C model code and maintains and serialises the data in the database. During the development, to ensure the data of the notebook is well organised, SQLite is used as the data storage solution. There are in-box strategies to use calling the Core Data library. In detail, firstly database is objectified by using NSPersistentStoreCoordinator method. After that the identified table can be requested from the database by calling managedObjectModel method so that the data of the table from database can be mapped to Objective-C code. To operate on the Core Data object, NSManagedObjectContext instance have to be created so that developer can send action and object to the certain instance. By doing this, the data between database and Core Data object is synchronised in the life cycle of a request.

4.4.3 Monkey Patching During the development, a stack data structure is needed to store the serialised event. At first, the stack is implemented by inheriting from the NSMutableArray, which is the mutable array class provided by Apple. The implementation was originally expected to be consisted by only a few lines of code, however the fact is that a lot of methods inside the NSMutableArray class need to be overridden. From the documentation from Apple, it strongly recommends that classes provided by SDK

28 should not be inherited, in case it will break the rule of ‘Mix-in over Inheritance’ and sometimes causes unpredictable result. The reason why inheritance is not recommended is simply because when other frameworks rely on the behaviour of the original methods, since the origin methods is overridden, it will certainly produce some chaos. However, by mix new extensions into the existing class, the previous behaviour of it will not be changed, so that the extended class will not have negative impact on the other library, which makes the legacy code easier to maintain. As the facts listed above, the implementation by inheritance is refactored to mix-in.

As the Objective-C programming language provided the functionality called ‘Category’ which can patch new feature to an existed class even the fundamental SDK, it is finally chosen to develop the stack for containing serialised events. In details, an interface named as EventStack is implemented as the extension of NSMutableArray class. After that, when NSMultableArray+EventStack.h is imported, the NSMutableArray will automatically contain the feature of EventStack interface.

4.4.4 Observer Pattern In software development, the objects that maintains a list of dependencies of other objects is called observer. When the state of the observer is changed, it will then notify the observed objects and trigger the states of them. The whole procedures can be regarded as the observer pattern. The observer pattern is quite useful to construct event-based software.

A typical usage of observer design pattern in this project is the text box resizing in iOS note application. When using the text editor for iOS platform, after the cursor focusing on the text view, the keyboard box will come out from the bottom of the screen, which will cover the certain area that is occupied by text view. Without text view resizing, the covered text area will stay covered till the keyboard box dismissed. With the growth of the note editing, when the text comes to the covered area, user cannot see the text that are being edited, which will bring very poor user experience. To solve this issue, the NoteViewController object is set to listen to the behaviour of the keyboard appearing and hiding. To ensure that the behaviour is text box resizing is non-blocking, notification centre is used as a message queue so that the every time the keyboard appear or dismiss, those events will be enqueued to the default notification centre and will be send to the controller object. As the NoteViewController has already selected - (void)keyboardWillShow:(NSNotification *)notification and - (void)keyboardWillDisappear:(NSNotification *)notification to handle the events of keyboard appearing and dismissing, The text area will simple shrink when the keyboard appears and enlarge to its origin size when the keyboard dismisses.

29

5 Performance Optimisation

As main functionality of the multi-device application is implemented, emphasis is then put on the performance optimisation. The computation intense and memory intense part of the application is discovered. Possible ways to optimise is then discussed. The performance is analysed by tools provided by Xcode 5 from all aspects such as CPU usage, thread loads and switches, memory usage etc. To dive into the details about the CPU and memory usage, the applications is profiled and analysed in instrument level. By analysing the stack, heap, virtual machine allocation, memory leaks and the running time of the applications, the bottlenecks can be found and possibly optimised.

As two applications shared the same Markdown parser as the core library, which is known as the Markdown parser, the performance of the parser is profiled at first. It is not optimised during development, as ‘premature optimization is the root of all evil20’. Multithreading technology is introduced so that the Markdown library can run on multiple cores. Some expensive calculation will be preloaded into static memory to improve the performance. In Section 6.3, the main procedures and result about how the parser is optimised is discussed.

Since the two applications are built for different devices, some of the performance issues might be platform-specific. On the other hand, as the architecture of OS X and iOS software development tool kit are quite similar, some performance bottleneck might still be similar. Caching technology and lazy loading will be introduced to reduce the unnecessary computations. Multithread technology will be applied to isolate intense computations from UI display to ensure the high responsiveness of the notebook application. In Section 6.4 and Section 6.5, the platform-specific bottlenecks, possible optimisation that solves the problem and result are discussed.

5.1 Profiling Approaches

Some approaches are used to profiling the performance of the application on both iOS and OS X platform. First one is the Debug Navigator provided by Xcode, which shows the CPU usage, memory usage and energy impact. When profiling the CPU usage, it demonstrates the workload per thread, which is extremely useful as multithreading is expected to apply to the application to improve the performance.

To profile the performance of each line of code, a tool named Instruments are introduced to the project, which shows the CPU and memory usage hierarchies that is related to the source code. This is to help to locate the code snippets that have performance issue. For example, the code snippets that cost huge amount of memory can be found by tracing the most allocated memory in the system monitor.

Manually profiling is also used in the project. As the nature of client side application, it makes no sense to profile the application by repeating an operation for several

30 hundred times. The most straightforward way of profiling is simply to use the note application on multiple devices. The performance of the program can be regarded to be acceptable as long as it does not crash and can run responsively and fluently on multiple devices. Moreover, timing functions can be added between the beginning and end of a code snippet so that the times a certain function takes to run can be easily tested.

5.1.1 Hardware for Profiling A MacBook Air is used to discover the performance of the Markdown parser, which contains an Intel Core i7-4650U Processor, 8GB 1600MHz LPDDR3 SDRAM and a 128GB PCIe-based flash storage. The selected Intel i7 processor has 2 cores and 4 threads. As each core has two virtual threads, the program run on it technically can do 4 jobs in parallel ideally, if the instructions of the parallel code are operating on separated data. The clock speed of the processor is 1.7 GHz however can reach 3.3GHz on max turbo frequency.21 The performance of Intel i7 cannot compare with modern HPC machine, however can reach acceptable parallelism for most of the client-side applications if the multi-core feature is implemented.

For iOS platform, an iPhone 4s is used to explore the performance of the application, which is consists of a dual-core ARM Cortex-A9 MPCore22, the clock speed of which can reach 800 MHz at maximum. Due the limitation of the number of cores and clock speed of the processor, regardless HPC machine, it cannot reach the performance of a laptop. As a result, the performance is expected to benefit more from concurrency instead of parallelism, by which the note application can be more fluent and responsive.

31

6 Result and Evaluation

After the application is implemented on multiple devices, most of the functionality is done. To test the usability of the application, it has to go through some use cases.

6.1 Application Run on OS X device

Following the design, the note on OS X is developed as is shown in Figure 7. By using Markdown as the mark-up language, user can easily output rich-text content without selecting varies of redundant buttons and options. The area of note editing is split into two parts. The left hand side is for note keeping. The right hand side is the preview of the notebook content, which converts the text written in Markdown syntax on the right-hand side to HTML format. By applying real-time rendering feature, every time the content in the left hand side is changed, the preview content will also change immediately. By clicking the Save button the content of the notebook can be then saved to the database, so that the data is persisted even if the application is closed.

Figure 7: The Area of Note Editing within Application on OS X.

Furthermore, a stand-alone notebook is not enough. To enable the people who are not using the application can also view the content of note, PDF exporting functionality is also provided. The content can be exported in PDF format, as is shown in Figure 8.

32

Figure 8: Exported Content in PDF Format.

6.2 Application Run on iOS device

Due to the limitation of the screen size and hardware performance, the notebook application on iOS devices is not as functional as the one run on OS X. As is discussed in Application Layout section, the auto-layout technology is introduced, so that the screen will be compatible for most of devices run on iOS. For the notebook application, it is tested on an iPhone 4s and a New iPad, which is proved to be compatible, as is shown in Figure 6.

To ensure quick note searching, the searching functionality is implemented as is shown in Figure 9. With advanced case-insensitive searching, even with a lot of notes stored in device, the certain content can still be found easily by typing in the search bar.

33

Figure 9: Searching Functionality on iOS Device.

Despite the performance limitation of iOS devices, they still have advantages such as high-resolution camera. The notebook application on iOS device integrates this feature. When the user is editing the note, he or she can attach a picture by directly taking one or uploading one from the photo library, as is shown in Figure 10.

34

Figure 10: Upload The Image and Display in The Notebook Application.

Thought the notebook application being developed can now run on multiple devices and is proved to be functional and useful, however the performance and responsiveness are not satisfied, since lags exist within the application. Some CPU and memory optimisation need to be done to improve the usability of the application further, which are discussed in next sections.

6.3 Markdown Parser Optimisation

The performance of the Markdown parser is quite essential to the project. Considering the real-time rendering feature in OS X note application, every time the text view is changed, the Markdown parser will render the view again, which not only uses memory to construct parse tree, but to recognise the patterns as well. Consequently, the parser is regarded as one of the most computation-intense part of the application. In next few subsections, the memory and CPU usages and possible improvement of the parser will be discussed.

6.3.1 Object Loading Optimisation As is discussed in the background sections, to reduce the cost spent on memory management, ARC is introduced to reduce the size of the code base. As a result, the memory management can be directly handled by executable itself. However even if automatic reference counting is used, improper memory usage might still exist in the program, since the strategies of memory management such as strong and weak object references are still managed by developer. If non-released memory accumulated to some extent, it will finally cause memory leak and crash the program, which absolutely should be prevented. For the purpose of optimisation, the memory usage of the parser is profiled.

35 The program for memory profiling is shown in Code Snippet 2. The test code does 10000 times of iteration for Markdown document with a relatively small scale. The content of the document in Markdown syntax is shown as the document object in the code. In each iteration of the loop, the document object is created, then a parser instance that parsing the content will be constructed and then parsing and rendering the view.

In case the variables within the loop block will be dereferenced between the intervals of the loop, its referenced memory will simply be released. However, when doing the profiling, the memory increases significantly. As is show in Figure 11, it takes 25 seconds and 948MB memory to do 10000 times of Markdown document parsing work, which indicates that some memory is not released as expected and some intense computation might exist in the program.

@autoreleasepool { NSDate *start = [NSDate date]; for (int i = 0; i < 10000; ++i) { Document *doc = [[Document alloc] initWithContent:@"## Hello world\nHello world\n---\nHello World\n\nefgh\n\n1. *abcde*\n- b\n* c\n\n---\n![http://image.com][your image]!v[https://www.youtube.com/embed/slNHuM4X-8M][500x400]"]; Parser *parser = [[Parser alloc] initWithDocument:doc]; [parser parse]; [parser render];

} NSTimeInterval timeInterval = [start timeIntervalSinceNow]; double seconds = fabs(timeInterval); NSLog(@"%lf", seconds); }

Code Snippet 2: The Memory Profiling Code of The Markdown Library

36

Figure 11: Original Memory Usage of the Markdown Profiling

To investigate the main memory usage of the application, the overall memory usage of the code is broken down into instruments. The benefit is that the bottleneck is not detected by guessing but analysing, which is proved to be timesaving method. The memory usage hierarchies in tree structure are shown as the Figure 12. From the figure, it is easy to detect that the main memory usage of the whole procedures is not the lexical analysis but code generation, as the –[Parser render] method occupies 90% of the memory. To investigate the true bottleneck further, the code is analysed one instrument by one instrument. Finally as is shown in the figure, the biggest bottleneck of the program is proved to be the –[BaseFragment replaceContextWithPattern:withFormat:] method, which belongs to the base class of most of the fragment class, is method to convert the Markdown text to HTML file, as is shown in Figure 13. As a result, the method needs to be optimised.

37

Figure 12: The Memory Usage Call Tree

In order to detect and optimise the bottleneck of –[BaseFragment replaceContextWithPattern:withFormat:] method, it is also broken down to instruments and being profiled. The main percentage of memory usage is shown in Figure 13. The figure shows that within the method, the memory generated by the regular expression instance and its replacement method is not released. This is mainly because the NSRegularExpression is not memory efficient, which means it cannot be managed efficiently by ARC technology. As is introduced in Mac Developer Library, “Regular expression matching in particular is an expensive operation.”23 What makes things even worth is that the program will create the NSRegularExpression object instance by calling regularExpressionWithPattern method every time the pattern recognise and recognise function is called.

Figure 13: Broken down instructions of replaceContextWithPattern method

In case the regularExpressionWithPattern method is both CPU and memory hungry and some memory used by this method can’t be released, lazy-loading strategies is applied to it. This means that the method is loaded in the compile time instead of loading over and over again in the running time, which can be implemented by declaring the regular expression created by regularExpressionWithPattern method as a static variable. As a result, the static variable will be only created once it is called. By doing this, the memory is allocated on the stack instead of in the heap, which is also regarded as a more efficient approach. 38 For the implementation in details, all the NSRegularExperssion objects existed in the code that are dynamically loaded have to be converted to static variable. As Objective C programming language has provided class method called initialize that can help loading static variables in the compile time. The program can be refactored as is shown in Code Snippet 3. The code snippet is an example of preloading, which lazy loads the regular expressions that are applied to recognise the patterns. This method is only called once as the class is referenced. On the other hand, if the header of the certain code snippet is not referenced in the program, then it even will not be evaluated in the initialisation, which greatly reduces the amount of computation and memory usage.

+ (void)initialize { if (!h1Regex) { h1Regex = [NSRegularExpression regularExpressionWithPattern:h1Pattern options:0 error:nil]; }

if (!h2Regex) { h2Regex = [NSRegularExpression regularExpressionWithPattern:h2Pattern options:0 error:nil]; }

}

Code Snippet 3: Lazy Load Regular Expression That Recognise Pattern

As the computation-intense and memory-hungry code is refactored, the program is compiled and profiled again. With the same piece of testing code, the profiling result is as shown in Figure 14. As is expected, it only takes 6 seconds to render the Markdown document for 10000 times, which is more than 4 times faster than previously. As to the memory usage, it takes 502 MB, which is only half of the usage of the program without optimisation.

39

Figure 14: Profiling after Lazy Loading Regular Expression Object

6.3.2 Multithreading Optimisation The first version of the Markdown parser is written in single thread, which means that it can only take the advantage of one core. As the most smart computers nowadays provide at least a dual-core processor, technically the Markdown parser can run at least 2 jobs on parallel. As some processors even support hyper-threading, each core might have 2 threads, which enable the program to achieve even better performance. Obviously the library does not take the advantage of high performance, multithreading is then applied to the parser library.

To profiling the program, different code is used, as is shown in Code Snippet 4. At first, the code generates a large document with around 110000 words, which can be regarded as a thick book. After that it runs a for loop with 10 iterations. In each iteration, the document in Markdown format will be parsed to HTML format. At last, the performance of the program is then profiled.

@autoreleasepool { NSMutableString *content = [[NSMutableString alloc] init]; for (int i = 0; i < 10000; ++i) { [content appendString:@"## Hello world\nHello world\n---\nHello World\n\nefgh\n\n1. *abcde*\n- b\n* c\n\n---\n![http://image.com][your image]!v[https://www.youtube.com/embed/slNHuM4X-8M][500x400]"]; } NSDate *start = [NSDate date]; for (int i = 0; i < 10; ++i) { Document *doc = [[Document alloc] initWithContent:content]; Parser *parser = [[Parser alloc] initWithDocument:doc]; [parser parse]; [parser render]; } 40 NSTimeInterval timeInterval = [start timeIntervalSinceNow]; double seconds = fabs(timeInterval); NSLog(@"%lf", seconds); }

Code Snippet 4: Profiling code that detect the concurrent performance.

Without any high performance technologies such as multithreading involve, The single thread program takes 66 seconds to finish the profiling, uses 4.55GB memory and 82% load of the CPU on average, as is shown in Figure 15.

Figure 15: Performance of Single Thread

Multithreading technology is then considered to be applied to improve the performance by taking the advantage of multiple cores. It is noticed that the parser is mainly consisted by lexical analysis, parsing and code generation. The procedures of lexical analysis and parsing cannot take the advantage of parallel, as they must be done line by line as in most case relations exist between lines. For example, it is hard to predict if a line is a paragraph or a header before reading the next line. Running this part of the program in multithreading environment simply will break the order of analysis and parsing. The most direct impact is that the grammar tree generated is disordered, so the HTML text generated by grammar tree is incorrect in most cases, which is certainly not expected.

41

Figure 16: Origin CPU and memory usage to run Code Snippet 4

On the other hand, it is possible to generate the HTML text from grammar tree in parallel. Considering the first level the grammar tree, which are elements such as paragraphs, lists, headings etc. In case the order of them is fixed, the code generation of each node can be done in multithread. After a certain node is generated to HTML text, the text can be save to a thread safe array. Eventually after all the nodes are generated to HTML and filled into the array, the HTML contents in the array can be joined so that the final HTML content is able to be generated.

According to the concept of multithreading, the program can be implemented in fork-join pattern, as is shown in Code Snippet 5. Firstly the program created a thread-safe array, which is used to store the generated HTML. After that, a task queue with high priority is created, which invokes the Grand Central Dispatch library within Apple ecosystem. During the iteration of the first level of grammar tree, each

42 Markdown node is dispatched to the task queue so that the HTML code generating work and HTML content storage is done in some other thread asynchronously. A barrier is set after the loop to ensure that the program will not run the string join method till all the multithreading work is done. Eventually the final rendered HTML is given by running componentsJoinedByString method.

-(NSString *) render { if (_renderedString == nil) { NSInteger elementsCount = self.document.elements.count;

NSMutableArray *arrayOfRenderedString = [NSMutableArray arrayWithCapacity:elementsCount]; for (int i = 0; i < elementsCount; ++i) { [arrayOfRenderedString addObject:@"Object-1"]; }

dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0); dispatch_group_t group = dispatch_group_create();

for (NSInteger i = 0; i < elementsCount; ++ i) { dispatch_group_async(group, queue, ^{ arrayOfRenderedString[i] = [self.document.elements[i] toHTML]; }); }

dispatch_group_wait(group, DISPATCH_TIME_FOREVER);

_renderedString = [arrayOfRenderedString componentsJoinedByString:@"\n"]; }

return _renderedString; }

Code Snippet 5: Concurrent Code Generating by Using GCD

After applying Join-Fork concurrency pattern and using the Grand Central Dispatch feature, the performance is as shown in Figure 17. Compared to the previous performance run on single thread. This time multi-cores are invoked to implement concurrency. As is shown in Figure 17, 4 threads run concurrently. The memory usage of the program declined significantly to 1.76GB compared to 4.55GB previously. The CPU usage is raised to 373% on average, due to four cores are almost fully loaded. However due to the reason of frequently switching among threads, the timing of the parser is not improved too much, which takes 56 seconds, 9 seconds faster than previous.

43 The performance of the program is then improved, however not as satisfied for several reasons. Firstly the frequent switches among threads causes large overhead, which is a large cost despite the lightweight nature of threads. Secondly, arrayOfRenderedString[i] = [self.document.elements[i] toHTML] procedure in the program has to access the shared memory among threads, which will cause the suspension of threads. To improve the performance further, the overhead caused by switches of threads and memory access should be reduced, which is discussed and improved in the next subsection.

Figure 17: Performance after using Multithreading.

6.3.3 Multithreading Overhead Reducing Emphasis is then put on the previous multithreading implementation of the code generation procedure for the Markdown parser. For the first level nodes of the document, their code generation jobs are dispatched to the task queue separately as is shown in Code Snippet 6. This inevitably causes some unnecessary switching among the threads. To prove it, the program is profiled in details that shows the threads and multi-core usage, as is shown in Figure 18. The workloads are dispatched to four cores (2 physical cores and 2 logical cores) evenly as expected. The load of each thread is also even, which is satisfied. However too much thread switching between threads. There are more than 155000 times of thread switching on average for each thread, which generate too much overhead. That can well explained why the performance is not improved significantly with the introduction of concurrency.

dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0); dispatch_group_t group = dispatch_group_create(); for (NSInteger i = 0; i < elementsCount; ++ i) { dispatch_group_async(group, queue, ^{ arrayOfRenderedString[i] = [self.document.elements[i] toHTML]; }); }

44

Code Snippet 6: Each code generation job is dispatched into the task queue separately.

Figure 18: Multi-Core and Multithreading Usage, as Well as Thread Switches.

To reduce the number of switches among threads, instead of sending each document node to the thread pool evenly, once sending a bunch of nodes to the thread pool should reduce the number of switching. Considering the number of cores of the processor is equal to N and the number of Markdown document nodes is equal to M. The number of nodes sent to each cores can be approximately as even as M/N. As the nodes are evenly arranged to the threads, there will be no thread switches during the computation if all the threads do not have too much dependence among each other. The Grand Central Dispatch is applied as the high performance model previously, which ensure that the program run in highly concurrent status. In case of the highly encapsulation nature the library has, it is up to the GCD to decide how much concurrency and parallelism is involved in the program, on the other hand, the amount of parallelism cannot be guaranteed. In general, the much parallelism involved, the better performance the program has.

The code is refactored as is shown in Code Snippet 7. As is discussed above, the number of threads is set the same as the number of cores. For each thread, the whole amount of code generation works is dispatched evenly to the threading pool. For example, if the number of threads is N and the number of works are M, then each thread will once get M/N amount of works. Consequently, each of the threads will not be switched due to job dispatching. 45 -(NSString *) render { if (_renderedString == nil) { NSInteger elementsCount = self.document.elements.count; NSInteger processorCount = [[NSProcessInfo processInfo] processorCount];

NSMutableArray *arrayOfRenderedString = [NSMutableArray arrayWithCapacity:elementsCount]; for (int i = 0; i < elementsCount; ++i) { [arrayOfRenderedString addObject:@""]; }

dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0); dispatch_group_t group = dispatch_group_create();

NSInteger dispatchingBlock = elementsCount < processorCount ? elementsCount : (elementsCount / processorCount); NSInteger startingElementIndex = 0;

while (startingElementIndex < elementsCount) { dispatch_group_async(group, queue, ^{

for (NSInteger i = startingElementIndex; i < elementsCount && i < startingElementIndex + dispatchingBlock; ++ i) {

arrayOfRenderedString[i] = [self.document.elements[i] toHTML]; } }); startingElementIndex += dispatchingBlock; }

dispatch_group_wait(group, DISPATCH_TIME_FOREVER);

_renderedString = [arrayOfRenderedString componentsJoinedByString:@"\n"]; }

return _renderedString; }

46 Code Snippet 7: Code Generation Run in Multithreading with Less Thread Switches.

The overall performance of the refactored code is shown in Figure 19. As is expected, the threading pool uses 4 threads to do the concurrency works, however the performance does not increase significantly, with only slightly timing and memory usage optimised. The amount of switches among threads of the program is reduced a lot, but not as much as expected.

Figure 19: Overall Profiling after Manually Scheduling

To explore the switching among threads in the program, multi-core monitor is again used to do the profiling. The result is shown in Figure 20. Compared to the previous automatically thread scheduling, the number of switches for each thread are reduced significantly, approximately 15000 times of switches fewer per thread, however does not change the result too much. As thread numbered 2, 3, 4 and 5 do the background code generation works, what should be noticed is that thread 4 and 5 are spawned by thread 3, which can explain why the low rate of parallelism. As is shown in the figure, the parallelism only occurred among thread 1 and 3. The relationships among thread 3, 4 and 5 are concurrency model, which means that they are run on the single core and are not able to run in parallel, but concurrently with intervals. This can well explain the switches among those threads.

Figure 20: Thread Usage after Manually Scheduling

Unfortunately the amount of parallelism and concurrency is all up to the GCD module to decide. Unlike traditional CPU machines that have some free nodes that are available to run specific computation work, the CPU of a smart computing devices

47 have to be scheduled to do other computations from time to time. CPU load of the Markdown library during the profiling is shown in Figure 21. The blue parts of the figure are the CPU loads that are occupied by markdown library. The blanked parts of the program show the idle and suspended status of the library. As the cores of a smart computing device not only have to run the Markdown parsing work, but are also scheduled to the work of other processes. As a result, the program has to be halted and continued frequently.

Figure 21: The Thread workloads are suspended frequently.

By optimising the memory usage and introducing multi-core concurrency and parallelism to the Markdown parser, the performance of the program increased hugely, which is expected. Initially when running the profiling program as is shown in Code Snippet 4, the program without optimisation uses 4.87GB memory and 243 seconds to finish, as is shown in Figure 16. After optimisation, it can be run within 55 seconds, with 1.75GB memory used. Applying multi-core technology to the program takes the advantages of the modern processors, however generates more heat and uses more energy, which is the trade-off. In case the usage of heat and energy is not that much in the real usage, it can simply be ignored.

As the Markdown Parser shared within multi-device applications is optimised, main emphasis is then put on the approaches to optimise the performance of the notebook applications on iOS and OS X devices, which is discussed in the following sections.

The library is then migrated to the cross-platform application. The core components now are running in the mix of concurrency and parallelism environment. However some platform-specific issues might exist in the applications. In next few sections, the issues discovered and how the performance is improved by solving them is discussed. As some concepts of performance optimisation between OS X and iOS platform might be overlapped, the overlapped optimisation discussed on OS X platform will only be discussed in brief instead of in details again.

6.4 Optimisation on OS X Platform

6.4.1 Reduce Unnecessary View Rendering The performance of most components of the note application on OS X platform is acceptable as the application can run smoothly in most cases. But the lags might occur when the user types words into the text area that originally contains a lot of words extremely quickly. The CPU usage of the application can be easily loaded to 97% as is

48 shown in Figure 22. Consequently, some approaches should be applied to optimise the performance of the note application.

Figure 22: CPU Usage of The Notebook without Optimisation.

The application applies real-time rendering, as is shown in Figure 7. The whole text in the text area has to be converted to Markdown parser immediately, which produces large pressure to the memory and CPU. Though the Markdown parser is optimised that can run on multiple cores, it is still an expensive computation. For example, when user is typing in the text area quickly, the CPU will suffer intense computation, every time the content in the text area changed, which is not necessary.

Considering the minimum lag that can be detected by a common human being is about above 60Hz response time, some rendering works can be just eliminated, as the application does not need to be too real-time. The strategies of rendering can be switched to rendering at a time interval instead of rendering every time the content is changed. If the time interval of the rendering is set at the range of 100ms to 300ms, the lag of rending might just seems unnoticeable to users.

-(NSTimer *) timer { double timeInterval = 2.0; if (!_timer) { _timer = [NSTimer timerWithTimeInterval:timeInterval target:self selector:@selector(renderContent:) userInfo:nil repeats:YES]; } return _timer; }

Code Snippet 8: Timer object that set to render the view at an interval of 0.3 second.

To implement this feature, an NSTimer object is set on the main thread of the application that set an interval of making changes at 0.3 seconds basic. To save the memory usage of the application, the timer is set as a lazy loading object. The implementation of the timer is as is shown in Code Snippet 8. As is set in program, the timer declares a method named renderContent, which is set as the callback of the timer. That means that the renderContent method will be called every 0.3 second. 49 Every time some changes are made within the text area, the content in it will not be rendered immediately. Instead, the event will be serialised and pushed to a thread-safe event stack. As is shown in Code Snippet 9, which is the implementation of the renderContent method. Every 0.3 second, the main thread will check if the stack has some item, if it is, the markdown document at the top of the stack will be popped from the stack, rendered as HTML text and finally displayed to the web view. The rest documents in the stack, on contrary, will be destroyed, as is shown in the certain code snippet.

-(void)renderContent:(NSTimer *)timer { if ([self.renderingStack isEmpty]) { return; }

NSString *content = [self.renderingStack pop];

Document *doc = [[Document alloc] initWithContent:content];

Parser *parser = [[Parser alloc] initWithDocument:doc]; [parser parse];

[[self.webView mainFrame] loadHTMLString:[parser render] baseURL:nil]; }

Code Snippet 9: View rendering at intervals.

After the optimisation, profiling is applied again to check the performance improvement. The result is shown in Figure 23. As the event stack and a 0.3 seconds timer are introduced to the application, the frequency of computation is significantly reduced, approximately 20% CPU usage on average, only 1/5 of the CPU usage of the program without doing any optimisation, which is satisfied.

Figure 23: Performance after Adding The Timer

50 6.4.2 Asynchronous Non-blocking Rendering As a timer and event stack is set to reduce the computation frequency, the performance is improved, since the CPU usage is declined and lags are eliminated while typing in a note with a large text. Nevertheless the performance can by improved further more. As the timing job runs on the main thread and the rendering job is triggered by timer, the lexical analysis work is still under-going in main thread. As the lexical analysis is a procedure with expensive computation, it will inevitably block the main thread for a while. Since the main thread mainly involves the GUI showing, when the display work is heavy, the expensive parsing computation potentially will cause lag within the user interface.

To enable the note application on OS X platform to be more fluent and highly responsive, the Markdown parsing work actually can be put on another thread so that it will not block the main thread. To implement that feature, GCD technology is again introduced for doing multithreading. The implementation is shown in Code Snippet 10. The Markdown document’s lexical analysis, parsing and generation work is simply dispatched to a thread pool rather than main thread. After the HTML generation work is done, the dispatched thread will then join back to the main thread, so that the UI can be responsible to user.

-(void)renderContent:(NSTimer *)timer { if ([self.renderingStack isEmpty]) { return; }

NSString *content = [self.renderingStack pop];

dispatch_async(self.renderingQueue, ^{ Document *doc = [[Document alloc] initWithContent:content];

Parser *parser = [[Parser alloc] initWithDocument:doc]; [parser parse];

dispatch_async(dispatch_get_main_queue(), ^{ [[self.webView mainFrame] loadHTMLString:[parser render] baseURL:nil]; }); }); }

Code Snippet 10: Non-blocking version rendering.

After refactoring the renderContent method, another set of profiling is done to test the performance improvement. The result is shown in Figure 24. In case the intense computation does not block the main thread anymore and runs on some other threads with the mix of concurrency and parallelism, the performance is improved 51 significantly. The CPU usage is proximately only 5% of the code without optimisation.

Figure 24: Performance after dispatching intense computation to run concurrently.

Finally, with 5% average usage of the CPU, the impact of the application does not have any pressure on the CPU. The performance eventually achieved 20 times improvement, which is acceptable.

6.5 Optimisation on iOS Platform

6.5.1 Multithreading Optimisation As the performance on OS X platform is optimised, emphasis is put on the improvement on iOS platform. The original version of the code run on single thread, it cannot take full advantage of Dual-Core feature of iOS devices. Figure 25 shows the performance of the original version of note application, for which the CPU usage cannot exceed 100%.

Figure 25: CPU Usage for The Original Version of Notebook on iPhone 4s.

Consequently, multithreading feature brought by GCD is introduced to the note application on iOS device. Like the note application on OS X device, the optimised Markdown Parser is also migrated to the iOS program. While doing some intense computation such as large text rendering, the application will take the advantage of Dual-Core so that the performance can be improved to some extent compared to single thread application. Due to the limited size of screen, the iOS version of note application does not have real time rendering feature, so that the application will not have as intense computation as the note application on OS X platform has. However

52 isolating the computation-intense Markdown parsing work out from the main thread might still bring benefits to the performance, as the Markdown rendering work sometimes will block the main thread. All the iOS devices till now only have a Dual-Core processor, which means that the application can’t reach satisfied parallelism. However by introducing multi-tasking, the computation-intense work can be dispatched from UI, so that the lags within the application can be eliminated. The refactored code is shown in Code Snippet 11. After the optimisation, the CPU usage is shown in Figure 22, which shows that the application is running on multi-cores, as the highest CPU usage can reach 112%. After the multithreading is introduced, there are not anymore noticeable lags in the application, which is acceptable.

MarkdownPreviewController *mpc = (MarkdownPreviewController *)segue.destinationViewController; dispatch_async(self.renderingQueue, ^{

NSString *title = [self getPlainTitle:self.note.content];

Document *doc = [[Document alloc] initWithContent:self.noteContent.text]; Parser *parser = [[Parser alloc] initWithDocument:doc];

[parser parse];

dispatch_async(dispatch_get_main_queue(), ^{ NSString *htmlContent = [NSString stringWithFormat:@"\n%@", [parser render]];

NSURL *baseURL = [NSURL fileURLWithPath:[[NSBundle mainBundle] bundlePath]];

mpc.title = title; [(UIWebView *)mpc.view loadHTMLString:htmlContent baseURL:baseURL]; }); });

Code Snippet 11: Dispatching The Markdown Parsing from main trhead.

Figure 26: CPU usage after multithreading optimisation. 53 6.5.2 Table View Optimisation Regarding the performance of the list of the notes in the application, which uses the table view and table view cell as the components on both OS X and iOS platform. Considering the situation of more than 1000 table view cells loaded on the user interface, for the note application on OS X device, in case the memory is plenty, it will not cause any issues. On the other hand, for iOS platform, table view will create a new UITableViewCell object every time a row is displayed when the user is scrolling the table. This is regarded as an expensive operation for both CPU and memory. That will have negative user experience when doing the scrolling operations. In real testing, lags can be strongly felt while scrolling the table view. The CPU usage is shown in Figure 27, which shows the performance by scrolling the table view from top to bottom and then from bottom to top with1000 UITableViewCell elements. As the component is created as is shown in Code Snippet 12, every time the UITableViewCell will be released when it is off-screen and be created when it appears. This causes huge amount of computations, as is shown in the figure.

Figure 27: CPU usage when scrolling the table view from top to bottom and then bottom to top.

UITableViewCell *cell = [[UITableViewCell alloc] initWithStyle: UITableViewCellStyleDefault reuseIdentifier:@"NoteTableViewCell"]; Code Snippet 12: UITableViewCell’s traditional allocation and initialisation.

To prevent the unnecessary CPU and memory usage within iOS devices, the table view cells should be reused. Considering the cells moved off-screen when scrolling the screen. Those cells actually should be put into a pool and cached temporary instead of being directly released. Then when a new cell is scrolled to be displayed onto the screen, instead of creating a new one, a used cell can be taken from the caching pool, combines with the data, and returns it to the table view. By doing this, the table view cell can be reused all the time. In details, instead of implementation like Code Snippet 12, the code is refactored as is shown in Code Snippet 13.

UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:@"NoteTitleTableViewCell" forIndexPath:indexPath];

54 Code Snippet 13: Dequeque a table view cell.

As the program is optimised, the profiling is shown in Figure 28. The CPU usage is reduced significantly. Moreover the application is more responsive and can now be scrolled without any lags. The user experience is then regarded to be improved.

Figure 28: CPU usage of scrolling after optimisation.

55

7 Conclusions and Future Work

One of the objectives of project was to investigate the state-of-the-art of developing responsive multi-device applications. The investigation was discussed in Section 2. It highlighted that the software should be developed natively in order to create a high responsive application utilising various high performance features provided within Apple ecosystem.

To prove the feasibility of multi-device development methodologies, a Markdown notebook application was developed using a methodology for multi-device application development. That is to build the common core functionality first, and then to implement the platform specific application simultaneously. As most of the functionality is developed, the approach of development is regarded to be useful.

Another important objective of the project was to create a highly responsive application by utilising the high performance high performance feature of Apple Ecosystem. To ensure the responsiveness of the application, memory and CPU usage of the application is tweaked on laptop and mobile devices. By introducing GCD technology, the application can exploit the parallelism and concurrency when multi-cores are available. Due to hardware and software limitations of mobile computing devices, the application cannot achieve as good parallelism as a traditional and dedicated HPC machine. However by taking the advantage of multi-cores, the application now can achieve parallelism and concurrency and run responsively. Moreover, by caching rows of table view, lazy loading UI objects, preloading expensive allocated object in the static memory, both CPU and memory usage is significantly improved.

On the other hand, the performance of the notebook application still has some flaws. Due to the black-box nature of thread pool of GCD, threads cannot be loaded as one thread per core, which would be the ideal case. It causes some inevitable and unnecessary thread switches. That certainly produces some overhead, however as threads are quite lightweight, the effect is limited. Another flaw is caused by web-view container. As the web view container run upon WebKit virtual machine, the memory usage within it cannot be managed. For the purpose of caching, the memory in web-view container applies garbage collection instead of ARC, which speedup the page loading, on the other hand, causes memory hogs.

7.1 Future Work

The data synchronisation is quite important to a multi-device application, A high performance web server for data storage is implemented, which is written in golang and using MySQL as the backend database. Due to limitation of time, the connection between the application and server is not built, which can be done in the future.

56 The local video uploading feature is also not implemented, mainly because it will cause memory hog in web-view container, since the size of videos is unpredictable. It can be solved by reduce the quality of video in the future.

As is discussed in Section 6.3.1, the regular expressions of the Markdown parser are both CPU and memory expensive. It is not only the nature of regular expression, but also due to the fact that the some of them are not written efficiently. Some investigation can be done to refactor some regular expressions to improve their performance.

To improve the performance of the note application on OS X platform, Posix Threads can be introduced to refactor the multithreading part of the program, as it can achieve better performance than GCD technology, though it is not encouraged.24

57

References

1 Martin Shen, Mobile-First or Web-First? Why Mobile Isn’t the Best Choice for Most New Startups. Online at http://technori.com/2013/06/4638-mobile-first-or-web-first-why-mobile-isnt-the-bes t-choice-for-most-new-startups/

2 http://www.netmarketshare.com/

3 Andre Charland, Brian LeRoux, Mobile Application Development: Web vs. Native. Online at http://dl.acm.org/citation.cfm?id=1968203

4 Apple Speeds Up WebKit’s JS Engine with LLVM JIT. Online at http://www.infoq.com/news/2014/05/safari-webkit-javascript-llvm

5 Abel Avram, Facebook: “Betting on HTML5 Was a Mistake”. Online at http://www.infoq.com/news/2012/09/Facebook-HTML5-Native

6 Lee Whitney, Why I Don’t Recommend Xamarin for Mobile Development. Online at http://www.whitneyland.com/2013/05/why-i-dont-recommend-xamarin-for-mobile- development.html

7 BlackDog Foundry, Creating a library to be shared between iOS and Mac OS X. Online at http://www.blackdogfoundry.com/blog/creating-a-library-to-be-shared-between-ios- and-mac-os-x/

8 Portability between iOS and OS X. Online at http://devcry.heiho.net/2012/02/portability-between-ios-and-os-x.html

9 Model-View-Controller. Online at https://developer.apple.com/library/mac/documentation/general/conceptual/devpedia -cocoacore/MVC.html

10 Automatic Reference Counting. Online at https://developer.apple.com/library/prerelease/mac/documentation/Swift/Conceptual /Swift_Programming_Language/AutomaticReferenceCounting.html

11 Migrating from Cocoa. Online at https://developer.apple.com/library/ios/documentation/miscellaneous/conceptual/iph oneostechoverview/PortingfromCocoa/PortingfromCocoa.html

12 http://clang.llvm.org/get_started.html

13 Clang vs GCC (GNU Compiler Collection). Online at http://clang.llvm.org/comparison.html#gcc

58

14 Grand Central Dispatch (GCD) Reference. Online at https://developer.apple.com/Library/ios/documentation/Performance/Reference/GC D_libdispatch_Ref/Reference/reference.html

15 About Threaded Programming. Online at https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/Multithre ading/AboutThreads/AboutThreads.html#//apple_ref/doc/uid/10000057i-CH6-SW2

16 Daring Fireball, Markdown. Online at http://daringfireball.net/projects/markdown/#acknowledgements

17 Github Flavoured Markdown. Online at https://help.github.com/articles/github-flavored-markdown

18 Design for iOS 7. Online at https://developer.apple.com/library/iOS/documentation/userexperience/conceptual/ mobilehig/

19 The Philosophy of UI Design: Fundamental Principles. Online at https://developer.apple.com/librarY/mac/documentation/UserExperience/Conceptual /AppleHIGuidelines/HIPrinciples/HIPrinciples.html#//apple_ref/doc/uid/TP3000035 3-TP6

20 “Premature optimization is the root of all evil” – DonaldKnuth. Online at http://c2.com/cgi/wiki?PrematureOptimization

21 Intel® Core™ i7-4650U Processor. Online at http://ark.intel.com/products/75114/Intel-Core-i7-4650U-Processor-4M-Cache-up-to -3_30-GHz

22 Cortex-A5 Processor. Online at http://www.arm.com/products/processors/cortex-a/cortex-a5.php

23 Performance, Using Predicates. Online at https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/Predicate s/Articles/pUsing.html

24 Avoid Creating Threads Explicitly, Design Tips. Online at https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/Multithre ading/AboutThreads/AboutThreads

59