Institutionen f¨ordatavetenskap Department of Computer and Information Science

Final thesis

Cross-platform Development for Wearable Devices

by

Gustav Beck-Nor´en

LIU-IDA/LITH-EX-A–15/038–SE

2015-06-10

Linköpings universitet Linköpings universitet SE-581 83 Linköping, Sweden 581 83 Linköping

Link¨opingsuniversitet Institutionen f¨ordatavetenskap

Final thesis

Cross-platform Development for Wearable Devices

by

Gustav Beck-Nor´en

LIU-IDA/LITH-EX-A–15/038–SE

2015-06-10

Supervisor: Magnus B˚ang,Link¨opingUniversity Niklas Bj¨ork´en,Accedo Broadband AB Examiner: Rita Kovordanyi, Link¨opingUniversity

Abstract

The market for wearable devices is continuously growing and has seen an in- crease in interest and demand this past year, specifically smartwatch devices. With several big players entering and trying to take place in the market the number of devices and platforms grow. This leads to device and software fragmentation like the one seen in the world of smartphones. In this paper I discuss and compare the two smartwatch platforms Android Wear and Apple Watch in terms of possibilities, limitations and differences. Research is done to find cross-platform development possibilities for these platforms. Extensive theoretical background of both APIs is researched and presented. An app for both smartwatch platforms is developed with integration of the WebSocket protocol to function as a remote control for a Video-On-Demand web service. This is done to showcase the cross-platform possibilities and differences of the platforms. As a result the biggest differences are out- lined and a conclusion is made that cross-platform development for these platforms can be challenging but is possible on certain levels.

iii Preface

I would like to begin by thanking Accedo Broadband AB for giving me the opportunity to perform this master thesis within such an exciting area. A special thanks to my Accedo supervisor Niklas Bj¨ork´enand also Niklas Lavrell for making me feel welcome at Accedo and helping me with feed- back and ideas during this thesis. I would also like to thank my examiner, Rita Kovordanyi, and my opponents, Oskar Eriksson and Emil Rydkvist, at Link¨opingUniversity. It has been very exciting working with smartwatches and it has been a real learning experience. Finally I would like to thank my family and my friends for the encouragement and support during this thesis work and during my studies at Link¨opingUniversity.

Gustav Beck-Nor´en Stockholm, June 2015

iv Contents

1 Introduction 2 1.1 Motivation ...... 2 1.1.1 Wearables ...... 2 1.1.2 Cross-platform ...... 3 1.1.3 Accedo ...... 3 1.2 Purpose and Aim ...... 4 1.3 Research questions ...... 5 1.4 Scope and Limitations ...... 5 1.5 Approach ...... 6

2 Theoretical Background 7 2.1 Smartwatches ...... 7 2.1.1 Challenges and Limitations ...... 8 2.1.2 Second screen ...... 9 2.2 Cross-platform ...... 9 2.3 Android Wear ...... 9 2.3.1 Google Play Services ...... 11 2.3.2 Architecture ...... 12 2.4 Apple Watch ...... 14 2.4.1 WatchKit ...... 15 2.4.2 WatchKit architecture and life cycles ...... 17 2.5 Accedo VIA Web ...... 19 2.5.1 MEAN Stack ...... 20 2.5.2 Architecture ...... 21

3 Method 23 3.1 Process ...... 23 3.1.1 Pre-study ...... 23 3.1.2 Implementation ...... 24 3.1.3 Evaluation ...... 24

v CONTENTS CONTENTS

4 Implementation 25 4.1 Web Server ...... 25 4.2 VIA Web application ...... 26 4.3 Android Wear ...... 28 4.3.1 WebSockets ...... 28 4.3.2 Native Android Wear ...... 29 4.3.3 Extended Notification ...... 31 4.4 Apple Watch ...... 33 4.4.1 WebSocket ...... 33 4.4.2 WatchKit App ...... 34

5 Result 37 5.1 Android Wear ...... 37 5.1.1 Native Android Wear ...... 37 5.1.2 Extended Notification ...... 38 5.2 Apple Watch ...... 40

6 Cross-platform Evaluation 42 6.1 Approaches ...... 42 6.1.1 Mobile web application ...... 42 6.1.2 Hybrid approach ...... 43 6.1.3 Cross compilation ...... 43 6.2 Possibilities ...... 44

7 Discussion 46 7.1 Result and Evaluation ...... 46 7.1.1 Differences and Similarities ...... 46 7.1.2 Challenges and Limitations ...... 48 7.1.3 Cross-platform Evaluation ...... 48 7.2 Method ...... 49

8 Conclusion 51 8.1 Future work ...... 52

A Web server 56

B VIA Web application 59

C Native Android Wear 64

D Android Wear Notification extender 73

E Apple Watch 81

vi List of Figures

2.1 Interaction pattern using a smartphone (Jordan, 2014) . . . .8 2.2 Interaction pattern using a smartwatch (Jordan, 2014) . . . .8 2.3 Google Play Services diagram (Google, 2015) ...... 11 2.4 High level architecture of Android Wear API ...... 12 2.5 The Data API ...... 13 2.6 The Data Layer API stack (Hahn, 2015) ...... 14 2.7 Communication between a WatchKit app and WatchKit ex- tension (Apple, 2015b) ...... 18 2.8 WatchKit app life cycle (Apple, 2015c) ...... 19 2.9 Accedo VIA (Accedo, 2015b) ...... 20 2.10 MEAN stack (Sevilleja, 2015) ...... 21 2.11 High level architecture of VIA Web ...... 22

4.1 Use case solution idea ...... 26 4.2 High level architecture of solution ...... 27

5.1 Final Native Android Wear Application ...... 38 5.2 Home screen with notification on wearable and phone . . . . 39 5.3 Final Android Wear Application ...... 39 5.4 Final Apple Watch Application ...... 40 5.5 Now Playing Glance on Apple Watch (Apple, 2015a) . . . . . 41

1 Chapter 1

Introduction

1.1 Motivation

This section covers some short background information about the topics and motivations to why this thesis was performed.

1.1.1 Wearables Wearable technology has been around for a very long time and is a very broad term but can be described as follows. A wearable device is simply a piece of technology that is incorporated in a piece of clothing or in an accessory that can be worn. This includes anything from pedometers em- bedded in shoes and trackers in bracelets, to something like microphone earrings. The wearable technology market had a short upswing and caught the normal consumer’s eye in the mid 80s with Casio’s wristwatch calcu- lator (Wikipedia, 2015b), but has been fairly absent from the consumer market since. Until now. The launch of Google Glass and the new trends in tracking our movements and habits has sparked a new interest in wearable technology. Healthier lifestyles have sparked the fitness tracker market and the need to be more connected now directs the consumer’s gaze towards the smartwatch. In May of 2012 the record-breaking (Kickstarter, 2015) success of the Kickstarter campaign for the Pebble Watch confirmed the indications that the demand for smartwatches is increasing. With technology getting more powerful and affordable the smartwatch, although still struggling, is finally starting to find its area of use and niche. Forbes writes that statistics from GlobalWebIndex says that 71% of those aged 16 to 24 would like to own a piece of wearable tech (smartwatch, smartband or Google Glass) (Lip- man, 2014). Google’s (2015) announcement of Android Wear in March of 2014 with partners such as Samsung, LG, Motorola, HTC and Asus, and the revealing of Apple Watch by Apple in September the same year makes it clear that non wants to miss the launch of the smartwatch market. With

2 1.1. MOTIVATION CHAPTER 1. INTRODUCTION all of the big players in the mobile industry diving into the area of smart- watches the innovation and number of devices available on the market is increasing rapidly. A report released in February by the market intelligence firm Tractica (2015) says that the ”Smart Watch Shipments Will More than Quadruple in 2015, Reaching 24.4 Million Units Worldwide”. This follows the smartwatch market forecast done by NextMarket (2013) which states the growth will continue year over year. The demand to wear a smartwatch on the wrist never been higher. With these new platforms launching on hardware devices that have not existed in this format before a lot of questions arises. As of starting this thesis work the Android Wear platform is fairly immature and Apple’s WatchKit is only in early Beta. Not much is known about the respective API’s and the limitations and differences between these new platforms. This is why this has to be thoroughly investigated.

1.1.2 Cross-platform In the world of mobile application development fragmentation between dif- ferent platforms and operating systems is becoming an increasingly large challenge for developers to overcome (Arthur, 2014). The term cross-platform refers to code, components or entire applications which can be run on on multiple different platforms. Having an application function on both the Android platform and the iOS platform simultaneously for instance is con- sidered cross-platform. This is no different in the world of wearables, specif- ically smartwatches. The smartwatch platform brings additional aspects into consideration with even more variety between models and brands than smartphones, with different form factors, functionalities and sensors. A key challenge for app developers across the world will be to ensure device com- patibility for maximum reach. This can be more complicated than it might seem at first glance. There are three major platforms today in the smart- watch market: Android Wear, Pebble OS and Watch OS (Apple Watch). But confusingly, even though LG and Samsung are listed as Android Wear partners, some of their devices are running their own operating systems. For example the LG Watch Urbane LTE is running an operating system based on WebOS and multiple Samsung devices are running a Tizen-based operat- ing system using the Samsung Gear SDK. The company Accedo Broadband faces this problem of platform fragmentation on a daily basis.

1.1.3 Accedo Accedo Broadband (henceforth referred to as Accedo) is a global company based out of Stockholm, Sweden and a market leading enabler of TV ap- plications (Accedo, 2015a). Accedo provides applications, tools and ser- vices to media companies, consumer electronics and TV operators globally, to help them deliver the next-generation TV experience. Accedo’s cloud- based platform solutions enable customers to cost-efficiently roll out and

3 1.2. PURPOSE AND AIM CHAPTER 1. INTRODUCTION manage application offerings and stores for multiple devices and markets. The company works with customers like Netflix, HBO, Viaplay and Disney. They have produced well known consumer video-on-demand applications e.g. HBO Nordic and Viaplay for various platforms. One of Accedo’s prod- uct solutions is called VIA. It is built to provide the customer with a highly customizable digital entertainment service and has the ability to deploy to multiple platforms (TVs, Game Consoles, mobile devices, the web and more). Accedo is always interested in the newest technology and in try- ing to find new use cases for these and incorporate them in their solutions and products. This is where smartwatches come into the picture. A paper written a the University of Oslo by Pradhan and Sujatmiko (2014) states that there is a value to and a demand for ways to minimize the interaction steps and effort required to perform certain tasks on the phone. The report suggests that the smartwatch can be a viable solution for this. This is some- thing Accedo sees value in. With many of their products already designed to function cross-platform e.g. VIA, what are the possibilities to extend these to the wrist of the user of every smartwatch platform? Are there any relevant and valuable use cases in a media consumption context? These are some of the questions they hope this thesis work can answer.

1.2 Purpose and Aim

This thesis work will focus its efforts within the area of wearables, specifically smartwatches. With the smartwatch market and number of devices growing, and facing the challenge of major fragmentation an investigation in cross- platform development for smartwatches will be done. To do this a smartwatch application for both the Android Wear and Apple Watch platform will be developed towards Accedo’s existing prod- uct solution Accedo VIA. Accedo explains this product as follows: ”Accedo VIA is a state of the art multi-screen application solution, which enables TV operators, media companies, and broadcasters to launch full-featured digi- tal entertainment services quickly and efficiently across multiple connected devices, including Mobiles, Tablets, Smart TVs, Game Consoles, and also the web. Accedo VIA provides a comprehensive set of reference applications that encompass all the major types of over-the-top (OTT) TV viewing” (Ac- cedo, 2015b). Multi-screen refers to the usage of multiple screens e.g. TV, web browser, smartwatch etc. in the same use case. One of Accedo’s biggest strengths today is their knowledge in a wide range of different platforms. With applications available across a large amount of different platforms it is relevant for them to investigate the possibilities in the new area of smartwatches. The VIA application is chosen as target for integration with smartwatches because by doing so this investigation becomes relevant and generates value for Accedo. They would like to include smartwatches in their product portfolio. The test application will be created cross-platform to illustrate a relevant key use case and be used for demonstration of the

4 1.3. RESEARCH QUESTIONS CHAPTER 1. INTRODUCTION projects progress. The web solution of VIA explained above will be the tar- get for this use case. The use case used during the entirety of the project is as follows: ”The user shall easily be able to control media playback (play/pause/vol- ume up/volume down) in the web browser containing VIA from his/her smartwatch”. Multi-screen in this use case is partly what drives the need for cross- platform functionality. I want to use one screen, the smartwatch, to control another screen, the web browser, and be able to do this for every smartwatch platform (Android Wear and Apple Watch). A solution to a similar use case is presented by Seettharamu Keshav et al. (2014) but their approach is to utilize the sensors in the smartwatch to produce a gesture based remote control for Smart-TVs. This thesis should provide insight to where abstractions to enable cross- platform development are relevant and possible across smartwatches. By implementing this relevant use case and integrating it with Accedo’s VIA Web, the pain points for cross-platform development should be discovered and solutions should be proposed. The finished thesis should also serve as a proof of concept for Accedo.

1.3 Research questions

With the thesis purpose and aim as a basis the relevant research questions can now be formulated. • Which are the integration points with the wearable devices APIs and external systems1, which can maximize portability and development synergies cross-platform? • Which are the similarities and differences between the smartwatch platforms from both a developer’s and a user’s perspective? By investigating and implementing a relevant use case using smartwatches in a media consumption context, the key areas where cross-platforms ab- stractions are possible and desirable will surface.

1.4 Scope and Limitations

Due to the time limitations of this thesis work, the broad subject of cross- platform development and the vast amount of devices encompassed by the term wearables limitations must be set to narrow the scope of the research. The following limitations will be followed throughout the thesis work: • Research and development will focus solely on the area of smart- watches. 1Outside the smartwatch e.g. companion phone or web.

5 1.5. APPROACH CHAPTER 1. INTRODUCTION

• Priority will lie on two smartwatch platforms: Android Wear and watchOS (Apple). • Only the use case stated in Purpose and Aim will be implemented and work as a proof of concept. • No major changes will be needed to the existing solution VIA save for integration of the communication with the smartwatch. • The aim is not to produce a complete and production ready application but to use the use case to find cross-platform possibilities.

1.5 Approach

As stated before, the thesis is performed at Accedo and is focusing on the area of smartwatches. During the thesis work the problem will be ap- proached from two different angles. Use case and literary studies, and a deep dive into the different wearable platforms and APIs as well as into Ac- cedo VIA Web. This will be done as follows: Firstly constraints and clear definitions of the concepts of wearables and smartwatches must be estab- lished. Then an investigation of relevant and universal use cases performed. Academic research regarding smartwatch interaction, design and APIs will be reviewed and presented. Challenges must be identified and considered during this phase. Next the test application realizing the aforementioned use case towards VIA Web will be developed on each platform as part of research in finding the common elements and integration points in the different APIs. The finished thesis should serve as a proof of concept for Accedo and provide answers to previously stated research questions. Evaluation will be done and conclusions will be drawn based of the results produced by the pre-study and implementation.

6 Chapter 2

Theoretical Background

This chapter introduces the reader to some basic concepts, techniques and theoretical background from research relevant to this report. Technical in- formation regarding both Android Wear and Apple Watch ecosystems is retrieved from each respective developer’s sites (manuals, API guides and references) unless otherwise stated.

2.1 Smartwatches

For decades the smartwatch has been a popular accessory in the world of science fiction as a gadget used for communication, tracking and control- ling other devices. In the real world is smartwatches in fact nothing new, but only recently is the technology and the functionality starting to match consumers expectation. There have been several attempts to get the con- sumer interested in smartwatches: Fossil’s wrist PDA and IBM’s WatchPad are examples of these (Rawassizadeh et al., 2015). Microsoft also made an effort in 2003 with the launch of the Microsoft SPOT (Albanesius, 2014). This gave the user access to feeds about news, sports, stocks, weather and more. This limited functionality was not enough to keep the users interested in wearing a bulky mini computer on their wrists. During the decade since, the progress necessary to intrigue the average user has been made in the field of wearables. Functionalities such as 3G/4G connectivity, advanced touch screens, voice commands, navigation and the need to feel even more con- nected than before makes the smartwatch highly desirable. The smartwatch are meant to give us the ability to stay more connected to the digital world, but more engaged in what is happening around us. This can be explained by the interaction patterns illustrated in Figure 2.1 and Figure 2.2. Figure 2.1 represents a user’s normal day interaction with their phone. Most smartphone users today can recognize their phone usage in this pattern agree that there is a lot of time wasted on interactions that should only take a second. This is where smartwatches can reduce the time overhead

7 2.1. SMARTWATCHES CHAPTER 2. THEORY

Figure 2.1: Interaction pattern using a smartphone (Jordan, 2014) per interaction by giving the user the right amount of information, ready for interaction, directly to the wrist. Figure 2.2 show the same set of interactions on a smartwatch.

Figure 2.2: Interaction pattern using a smartwatch (Jordan, 2014)

This is what Timothy Jordan, developer advocate at Google, meant by saying: ”The user’s more present in the real world, and yet more connected to the virtual world.” (Jordan, 2014)

2.1.1 Challenges and Limitations It is very important to recognize the limitations and challenges associated with smartwatches and not go overboard and make it out to be something it is not. When developing applications and searching for new ways to utilize the smartwatch platform one has to rethink the design and interaction patterns used for smartphones. Currently smartwatches are not stand-alone devices but rather an extension of our smartphones to make interactions easier and more accessible. Some notable challenges and limitations: • Very limited screen size. • Limited computing power. • Currently dependent on companion smartphone for connectivity and computing power. • Finding relevant use cases. • Designing applications using only micro interactions. • Immaturity of the platform.

8 2.2. CROSS-PLATFORM CHAPTER 2. THEORY

2.1.2 Second screen The term second screen refers to a device which purpose is to enhance the viewing experience that is taking place on another device. This can for example be a user’s smartphone or tablet working as a companion device (second screen) to for example a TV-set. Since the smartwatch currently more or less acts only as an extension, a second screen, to the companion smartphone it is relevant to look at how the smartphone itself is used as a second screen to other devices. This could be seen as one step further in the area of companion devices. The smartwatch act as an additional companion device to other devices by extension through the smartphone, a sort of third-screen. By considering how the smartphone acts as a second- screen interesting areas for the smartwatch can be found.

2.2 Cross-platform

As stated briefly in the introduction many of the challenges faced with cross- platform compatibility with smartphone application development are the same for smartwatch development. The fragmentation of mobile operating systems is due to the fact that all major actors on the market has developed their own non-standard methods for developing towards their platform. This includes using different operating systems (using unique programming lan- guages and SDK’s) and different hardware configurations making the user experience and user interaction different from platform to platform. Fur- ther fragmentation exists within the various platforms with applications not being able to run on certain different software versions of said platform. Limited development resources often lead to developers having to choose which specific platforms, devices and operating systems to target for the applications. There are several different development tools available to handle this problem for smartphones. These tools support various amounts of platforms and approach the problem of cross-platform development from different an- gles. According to Ohrt and Turau (2012) there are three main approaches to cross-platform development: mobile web applications, cross-compilation to separate native code and a hybrid approach using the web-view element on each platform. All these approaches and techniques will be taken under consideration when investigating the smartwatch platforms. A presentation and evaluation of these in a smartwatch context is available in the chap- ter Cross-platform Evaluation 6.

2.3 Android Wear

Android Wear is a version of Google’s operating system Android that is designed to run on smartwatches and other wearable devices. It was an- nounced along with the release of a developer’s preview on March 18, 2014.

9 2.3. ANDROID WEAR CHAPTER 2. THEORY

Since Android Wear extends the Android operating system much of the same functionalities are available, including Google Now, but with a differ- ent form factor. Much of the user experience of Android Wear is centered around Google Now. Supplying the user the right information, at the right time, using contextual information such as location, time and events. And giving the user the ability to see this information just at a glance. As al- ready mentioned due to the limitations of the smartwatch form factor user interaction is focused solely on micro interactions. Wearable devices run- ning Android Wear needs a companion Android smartphone with Android version 4.3 or later to communicate with. There are differences that need to be considered when developing ap- plications for Android Wear in comparison to Android. First of all there is a sleep timer that the system enforces. This means that the device will go to sleep if the user does not interact with the displayed activity. When the device wakes back up the home screen will be displayed, not the pre- vious active activity. And the difference that cannot be stressed enough is the limited size, both screen and functionality wise, a wearable application has. Generally the wearable application will contain only a subset of the functionality provided by the corresponding app on the handheld device. Consequently, the term micro interactions and supplying the user with the right amount of information, at the right time, once again is important. An- other noteworthy difference is the way apps are downloaded and installed in Android Wear. The Android Wear device cannot access the Google Play store so wearable applications need to be bundled together with a handheld application. When the handheld application is installed the system auto- matically installs the application on the wearable. However this handheld companion application can also be useful for handling heavy computation, performing network operations or performing other tasks and sending the results to the wearable. In addition to the limitations and challenges stated earlier regarding smartwatches, there are also certain limitations where some API’s available in Android simply are not available in the wearable device. The following API’s are not supported for Android Wear applications:

• android.webkit - Provides tools for implementing browsing of the web.

• android.print - Provides all classes and abstractions for implementing print support. • android.app.backup - Contains the backup and restore functionality available for applications. • android.appwidget - Components necessary to develop application wid- gets. • android.hardware.usb - Provides support to communicate with USB hardware that are connected to Android devices.

10 2.3. ANDROID WEAR CHAPTER 2. THEORY

The absence of these API’s puts some limits on the scope of what a wearable app can be and also limits the cross-platform possibilities explained in the Discussion chapter (7).

2.3.1 Google Play Services Google Play Services is a background service and an API package that let’s applications access Google-powered features such as Maps, Google+ and also Wear. The Google Play service client library contains interfaces that give the developer access to the individual Google services. That is the client library is used to interact with the Google Play service APK (Android application package) that runs as a background service in the Android OS (Figure 2.3). The Google Play store automatically distributes the Google Play services to all devices running Android 2.3 (Gingerbread) or later, and keeps them updated, which frees the developer from worrying about device support. To make a connection to one of the API’s provided by Google Play services an instance of GoogleApiClient (Google API Client) must be created. This serves as a common entry point to all services and manages the connection between device and service. Once connected, the client can make read and write calls using the service-specific API that the application has authorized, as specified by the API and scope added to the GoogleApiClient instance.

Figure 2.3: Google Play Services diagram (Google, 2015)

The use of the Google Play services is not required to extend notifications from handheld applications. It is used to send and sync data to and from the wearable/handheld with the Wearable Data Layer APIs. All Android Wear specific APIs are described in detail in subsection 2.3.2.

11 2.3. ANDROID WEAR CHAPTER 2. THEORY

2.3.2 Architecture Android Wear supplies the developer with a high level of abstraction and a set of APIs to handle operations. These APIs consist of the Message API, Node API and Data Layer API. They handle all the communication needed between devices in addition to the Notification API which can provide the wearable with notifications from existing handheld applications out of the box. The high level architecture of the communication between handheld and wearable and the API’s used is illustrated in figure 2.4.

Figure 2.4: High level architecture of Android Wear API

Node API The Node API in Android Wear handles events related to local nodes and nodes connected to the Android Wear network. A node in this context is simply a Android or Android Wear device. It tells the developer about which nodes exists in the network and gives information about these. It is used to identify and list available nodes during communication.

Data Layer API The Data Layer APIs allow for automatic data synchronization between nodes. This works by allowing the handheld and the wearable to take on the role of data sender, data receiver or both. A DataItem is a base object of data stored in the Android Wear network and is synchronized across all nodes in the Android Wear network automatically. DataItems can even be sent to non-connected nodes which will become synchronized one they come online. The DataItem itself consists of two different objects: the payload

12 2.3. ANDROID WEAR CHAPTER 2. THEORY object, which contains the actual data, and a path object which acts as a unique identifier string for the DataItem. The DataItem is only meant to send smaller batches of data back and forth between handheld and wearable. For larger amounts of data, e.g an image, an Asset object can be attached to the DataItem. The large amount of data is still sent over the bluetooth connection and the handheld device does all the heavy lifting, e.g resizing of said image. A useful feature with asset objects is that they can handle caching of data, which prevents retransmissions of objects and preserves computing power, battery and bluetooth bandwidth. The Data Layer API also contains the Message API which is described below. How the DataItem with an asset object is represented is illustrated in Figure 2.5.

Figure 2.5: The Data API

Message API The Message API exposes an API for components to send messages to other connected nodes in the wearable network. Messages are by default sent in the remote procedure call (RPC) type. That is, once the sender has sent the message there is no validation that the message was actually received at the other end. If this is not good enough messages can be set up to use the Request/Response model. The Message API is used to do common tasks like e.g telling the wearable to start a new activity, or tell the handheld to skip to the next song. Once again only small amounts of data should be sent as payloads through the Message API, for larger amounts use the aforementioned Data API using assets. Since the Message API is limited to sending messages to connected nodes the Data API is also used for sending messages that is to be delivered on node connection. Figure 2.6 shows an example how a message sent from the handheld to the wearable travels through the API-stack from device to device.

13 2.4. APPLE WATCH CHAPTER 2. THEORY

Figure 2.6: The Data Layer API stack (Hahn, 2015)

Notifications API The notifications in Android Wear works right out of the box. The wearable and the companioned handheld devices automatically share notifications. Notifications that appear on the handheld device appears as a card in the wearable devices context stream. To enrich the user experience develop- ers can add wearable-specific functionality to the notifications. This can be functions such as, actionable notifications, receive voice input, adding additional pages and stacking multiple notifications. Limitation: ongoing notifications will not show up on the wearable, these needs to be done manually on the smartwatch. An ongoing notification is a type of persistent notification which cannot be dismissed by a traditional swipe motion. This type of notification is always on top and can be useful for e.g. media applications for control of playback.

2.4 Apple Watch

Apple Watch is Apple’s smartwatch device and was first announced on the the 9th of September 2014 and later released on the 24th of April 2015. The Apple Watch is the first new device added to Apple’s product range since the iPad was released 2010. The Apple Watch has the most features one would expect in a smartwatch in 2015 but features some unique additions which demands some specific UI patterns. The most distinctive and, for now, unique feature of the Apple Watch is the digital crown. The digital crown is a dial placed on the side of the watch which, when turned, lets the user scroll through content, zoom and when pressed navigates back to the home screen. This allows for precise scrolling and zooming interaction on the watch. Another platform unique feature available on the Apple Watch is “force touch”. This technology allows the touch display to distinguish

14 2.4. APPLE WATCH CHAPTER 2. THEORY a soft screen-press from a hard one and consequently implement different functionality for each event. This is some compensation in the absents of multi-touch functionality in Apple’s SDK which is available on the Android Wear platform. Other features available on the Apple Watch are: speakers, Wi-Fi connectivity, NFC and haptic feedback (precise vibrations from no- tifications and user interaction). The Apple Watch uses Apple’s operating system designed for the watch called watchOS. To develop applications for watchOS developers are provided with WatchKit, a software development kit for Apple Watch. This section will explain the design, functionalities and limitations of the Apple Watch platform.

2.4.1 WatchKit WatchKit is the API and SDK provided by Apple to use for development of Apple Watch applications as well as for extending existing iOS applications to the watch. watchOS is the operating system, based upon iOS, which runs on the Apple Watch. It is developed by Apple and shares much of the func- tionality and design of iOS but with some limitations. It is designed around smaller simpler interactions and gestures to suit the Apple Watch’s interface better. WatchKit provides three different ways for the user to interact with and developer to extend applications: apps, glances and notifications.

WatchKit Apps The first thing that needs to be established is that third party apps for the Apple Watch are, as of writing this report, not stand alone applications. That is, they need a paired companion iPhone to function at all. Initially WatchKit apps are meant to extend and/or complement the iOS apps, not replace them. In addition to the obvious limitations in the form of screen size and user interactions etc. there are other challenges that need to be considered when developing WatchKit apps. Developers are for instance not able to access the sensors of the Apple Watch when developing third party apps. The range of unavailable sensors includes the gyroscope, ac- celerometer, heart rate sensor and NFC. The build in speakers, microphone, the Taptic Engine and the functionality of the digital crown are unavailable in WatchKit as well. WatchKit apps always consist of two parts: the WatchKit extension which runs on the user’s paired iPhone and a set of interface resources which are installed on the Apple Watch. With the limited number of styles and interfaces in WatchKit there are only two basic types of navigation available: hierarchical style and page-based style. Both navigation styles closely resemble their iOS counterparts but cannot be combined. The limited number of UI components available is due to the fact that WatchKit does not make use of the UIKit framework used for iOS apps. Here Apple has replaced controllers and elements with WatchKit-specific classes. Classes such as UIViewController, UITableView and UIButton for example are replaced by

15 2.4. APPLE WATCH CHAPTER 2. THEORY

WKInterfaceController, WKInterfaceTable and WKInterfaceButton. These new interface objects are a lot alike the ones in the UIKit framework except that they are more lightweight, and not as customizable. One UIKit view of importance from a cross-platform perspective that is not available in Watchkit is UIWebView. Other notable differences from iOS development are the life cycles of the WatchKit applications and their inability to run as background services. Both these topics are described in more detail further into the report.

Glances Glances on Apple Watch are meant to provide the user with short, useful information only meant for reading. They could be seen as feed of contex- tually relevant information and a lightweight view, a ”glance”, of the user’s WatchKit applications. Glances are accessible from anywhere on the watch by doing a swipe-up gesture from the bottom of the screen. They are not scrollable and cannot contain any other action than launching the app it represents. A glance is not required but limited to one per WatchKit app.

Notifications Apple Watch can make full use of notification interactivity supported in iOS, both remote and local. So if the containing iOS app supports notifications extending them to the Apple Watch only requires an implementation of the notification interface in WatchKit. The notifications are meant to provide the user with small amount of information at an appropriate time and can contain small lightweight actions. The user, if they choose to view notifi- cation, can decide to dismiss the notification, launch associated WatchKit app, or interact directly on the notification by touching the provided action button. There are two types of interfaces for notifications: the short-look in- terface and the long-look interface. The short-look interface tells the user that a notification has been received and gives only a very brief view of the notification. The short-look interface is not scrollable and cannot be customized. It always displays the app icon, app name and the title string from the notification. If the user interacts with the notification or simply continues to look at the notification (keeps their wrist raised) the watch will transition into the long-look interface. The long-look interface is scrollable, displays more detailed and comprehensive information about the notifica- tion and it is somewhat customizable. If a custom notification interface is not provided, Apple Watch displays a default interface that includes the app icon, the title string of the notification, and the alert message. If a custom notification interface is provided, this will be displayed instead. The layout of the long-look notification interface is divided into three different areas: • The system-provided sash at the top is an overlay that contains the app name and app icon. Only the color is configurable.

16 2.4. APPLE WATCH CHAPTER 2. THEORY

• The app content area contains the detailed information about the in- coming notification. • The bottom area contains the app-defined action buttons from by the containing iOS app and the system-provided dismiss button.

User interaction with notifications works as follows: Tapping the app icon in the sash will launch the WatchKit app. Tapping one of the action buttons in the bottom area will start the linked action on either the WatchKit app or the corresponding iOS app. Since the Apple Watch is limited to only running foreground actions any background actions will be offloaded to the iOS app. The dismiss button simply closes the notification without any further actions. Tapping anywhere else on the notification does nothing.

2.4.2 WatchKit architecture and life cycles As previously stated the WatchKit app runs as an app extension on the users iPhone device. App extensions gives the user access to content and functions from apps running on iOS 8 or Mac OS X Yosemite. Leverag- ing the extensions functionality in iOS 8 the iPhone contains and runs the code while the watch contains the application storyboard. This means that the watch always needs to be connected, via Bluetooth or Wi-Fi, to an iPhone for the app to function. This is the situation in the initial release of WatchKit but Apple (2014) has stated that third-party developers will be given the ability to develop native applications for Apple Watch in the future. Figure 2.7 illustrates how the architecture of the extension looks and how communication with the watch functions. The sharing of data and how the devices communicate is described more in detail in the section below. Although UIKit is unavailable in WatchKit the framework still conforms to the MVC (Model-view-controller) design pattern. As mentioned previ- ously the UIKit classes have been replaced by watch-specific ones. Each scene in a WatchKit app’s storyboard is managed by a single interface con- troller object in the form of an instance of the class WKInterfaceController. Consequently only one interface controller can be displayed on-screen at any given time. With the limited screen size available the system handles layout of UI-elements automatically and most often stacks them from top to bottom. This can be slightly customized by the usage of Groups that can put elements side-by-side in the top-down hierarchy. The life cycle of a WKInterfaceController is fairly straightforward, see figure 2.8, and does not allow any long-running task without user interaction. This leads to the deactivation and suspension of the app. The sharing of common data between the WatchKit extension and the containing iOS app is done through a shared app group. An app group is a secure container that multiple processes can access and is common practice when developing iPhone extensions. This makes it possible for WatchKit and the iOS app to share files and user defaults. Apps that work even more

17 2.4. APPLE WATCH CHAPTER 2. THEORY

Figure 2.7: Communication between a WatchKit app and WatchKit exten- sion (Apple, 2015b) closely with their containing iOS app can use a method that sends messages using a request/response model. This enables the watch app to run activities that require extra time to complete or be run in the background by running them on the iPhone. This is done by sending a request to the phone, which starts the task, and the result is then communicated back to the WatchKit extension when complete. The app delegate in the containing app performs the request using a provided dictionary and then returns a reply to the WatchKit extension.

18 2.5. ACCEDO VIA WEB CHAPTER 2. THEORY

Figure 2.8: WatchKit app life cycle (Apple, 2015c)

2.5 Accedo VIA Web

Accedo VIA is one of Accedo’s product solutions. It is built to provide the customer with a highly customizable product and the ability to deploy to multiple platforms. Accedo markets their VIA product as follows: ”Ac- cedo VIA is a multi-screen application solution that allows TV operators and media companies to launch full-featured digital entertainment services quickly across multiple connected devices, Mobiles, Tablets, Smart TVs, Game Consoles and the web.” Key features of Accedo VIA application: • Content Promotions

• Live TV • Program Guide • Video on Demand

• Search • Settings • My Account • Recommendations

• Favorites • Social (Accedo, 2015b)

19 2.5. ACCEDO VIA WEB CHAPTER 2. THEORY

Figure 2.9: Accedo VIA (Accedo, 2015b)

The Accedo VIA Web application is built on a set of JavaScript frame- works that together are referred to as the MEAN-stack (Figure 2.10). The MEAN-stack consists of three JavaScript frameworks together with a NoSQL database solution and they are all described briefly below. The MEAN ab- breviation is a contraction of the framework names MongoDB, Express.js, Angular.js and Node.js.

2.5.1 MEAN Stack Node.js Node.js is a JavaScript platform which enables very low effort building of fast, scalable web applications. It is a back-end framework which uses a event-driven architecture and it is very lightweight. The scalability and platform independence makes it well suited for a application of this type.

Express.js Express.js is a web application framework for Node.js. It helps the developer to organize the project into a MVC pattern on the server side and can help build single-page applications. Like Node.js it is very lightweight and has a low effort installation process. It is a tool for handling routes, requests and views in the Node.js back-end.

AngularJS AngularJS is also a JavaScript framework for web applications but functions as the client side front-end. It also uses the MVC pattern and adapts to and

20 2.5. ACCEDO VIA WEB CHAPTER 2. THEORY lets the developer extend the HTML vocabulary when developing single-page web applications.

MongoDB MongoDB is a cross-platform document-oriented NoSQL classified database. It enables queries to be made in JavaScript and data provided in JSON format which makes it ideal to work together with the rest of the frameworks.

Figure 2.10: MEAN stack (Sevilleja, 2015)

2.5.2 Architecture The high level architecture for Accedo VIA Web is built up as illustrated in figure 2.11. The figure (2.11) shows how the app, built using the aforemen- tioned MEAN-stack, runs from a high-level perspective. The VIA Web-site, is a Single-Page-Application web app built with Angular.js. It is rendered in the end-users browser after continuously fetching data, in JSON format, through AJAX requests to the server-side back-end component built with Node.js. The back-end Node.js app is responsible for integrating with 3rd party APIs. OVP is an abbreviation of Online Video Provider and AppGrid is Accedo’s app management solution.

21 2.5. ACCEDO VIA WEB CHAPTER 2. THEORY

Figure 2.11: High level architecture of VIA Web

22 Chapter 3

Method

This chapter briefly describes the method and processes used when carrying out this thesis work.

3.1 Process

Preferred by myself along with Accedo all of the stages during the thesis work was done iteratively following the agile methodology. As mentioned before in the Approach section in the Introduction the work was preformed in four phases. The pre-study (and analyzing the results), implementation and evaluation. These in turn where divided into sprints to more easily track the progress of the work and follow the time constraints. The high level view of the phases in which the work was performed is as follows:

• Perform pre-study research • Analyze the pre-study results • Start implementation – Specifying the use case and requirements – Solution architecture and design – Code and implement solution • Evaluate the implementation results

3.1.1 Pre-study Initially the pre-study consisted of researching and familiarizing myself with the concept of wearables. Defining the limitations of the thesis work and gaining an overview of the topic at hand was the first step of the thesis work. The later pre-study was performed with two different approaches: literary

23 3.1. PROCESS CHAPTER 3. METHOD research to find related work and a deep dive into the device APIs and manuals. The results of the pre-study and information needed to understand the final results, discussions and conclusions is presented in the Theoretical Background chapter.

3.1.2 Implementation With the necessary amount of knowledge acquired during the pre-study the implementation portion of the work could begin. The implementation part of the thesis work was a quite large and important part. By working extensively hands-on with development on the smartwatch platforms knowledge of the APIs and their differences could be discovered efficiently. As shown above the implementation phases begun once an analysis of the pre-study had been performed. With the theoretical background as a foundation the solution design and architecture was done followed by the actual implementation. The phase started with the Android Wear platform. This because of the fact that WatchKit was still in early beta at this stage. I wanted to hold off with the development for Apple Watch hoping for further updates to the SDK. This development process is presented in its entirety in the Implementation chapter.

3.1.3 Evaluation With the implementation completed along with the deep theoretical knowl- edge about the platforms a relevant and qualified evaluation of the work could be performed. A high level discussion and concrete conclusions is be presented in the Discussion and Conclusion chapters.

24 Chapter 4

Implementation

This section showcases the implementation of the use case previously ex- plained in the report. It includes an exposition of the planning and design process as well as the development using the technologies from the theo- retical background. Conclusions and differences between the process and implementation of the different platforms are discussed in chapter 7. To reiterate, the target use case used for the investigation during this thesis work is as follows: “The user shall easily be able to control media playback (play/pause/vol- ume up/volume up) in the web browser containing VIA from his/her smart- watch”.

4.1 Web Server

The design and implementation of the environment outside of the smart- watch app context will be explained. To enable communication between the smartwatch app and the Accedo VIA Web application a server was needed as a go-between. This was set up using the Node.js framework. The sim- plicity and low effort to set up a server and add a WebSocket library made this the optimal platform to work with for this thesis work. Additionally is the VIA application back-end built with the same framework which limited the scope of frameworks and languages used during development. Figure 4.1 illustrates a high level view of the solutions idea for the use case. Once the server was up and running an implementation of the WebSocket protocol was added to both the server and VIA Web application (Figure 4.2). The WebSocket protocol is an independent TCP-based protocol that enables full-duplex communication over a single TCP-connection. After a hand- shake has been performed between client and server it uses a stream of messages in real time which can be associated with event listeners to handle these. The implementation of WebSocket protocol on the different platforms will be explained within this chapter. Some further modifications was also

25 4.2. VIA WEB APPLICATION CHAPTER 4. IMPLEMENTATION

Figure 4.1: Use case solution idea required in the VIA Web application to handle the events in both directions when sending and receiving WebSocket messages. This was necessary to be able to control the media playback and also to keep the smartwatch and web application in sync. A WebSocket implementation for Node.js called WebSocket-Node devel- oped by McKelvey (2015) was used. This enables the developer to require the library and start the WebSocket server as shown in Appendix A.1. The server requires connecting devices to identify themselves when the connec- tion opens. This way the server can keep track of which node is the web application and which is the mobile device and relay messages accordingly. Messages are configured to use the JSON format across all platforms to be able to bundle more information within each message. The implementation, how the messages are handled and forwarded to the right device is shown in Appendix A.2.

4.2 VIA Web application

Since the current VIA Web application does not implement the WebSocket protocol this had to be added to enable remote communication to and from the app. With the massive code base that is VIA I looked for a minimum effort way of doing this. With this in mind I decided to implement the WebSocket as a service in the front-end of application. The VIA Web ap- plication is built on the previously explained MEAN-stack so a WebSocket

26 4.2. VIA WEB APPLICATION CHAPTER 4. IMPLEMENTATION

Figure 4.2: High level architecture of solution library for AngularJS was required. I decided to use a WebSocket library de- veloped by Ferrari (2014) called ng-websockets. This is a WebSocket library specifically for AngularJS and can be installed right into the application via the package and dependency manager Bower (Appendix B.1). The service, webSocketService.js, handles the communication with the server. It is im- plemented into the video element in VIA in a way that starting a video opens up the WebSocket connection. The service sends a message identifying itself as a browser when the connection opens. The videoplayer registers listeners to the webSocketService and is now ready to receive and send messages. webSocketService is available in Appendix B.2 and the video player element in Appendix B.3. The WebSocket messages are configured to be sent in the JSON format to bundle more information in each message. This way in ad- dition to the desired command meta data such as title, description and URL to the cover image of the media being played could also be delivered to the smartwatch. How the messages are handled by the different mobile devices is explained under their respective chapters. Listing 4.1 shows an example of how a typical message from the browser client VIA Web application to the server could look.

27 4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION

Listing 4.1: Example of JSON formatted WebSocket message sent from the browser { ”event”:”webPlay”, ” data ” : { ”poster”:”http://ovp.cloud.accedo.tv/images/dvd cover /616 a66736cd781b1293c7b9b7c739627 . jpg”, ”title”:”Interstellar”, ”description”:”A group of explorers make use of a newly discovered wormhole to surpass the limitations on human space t r a v e l and conquer the vast d i s t a n c e s involved in an interstellar voyage.” } }

4.3 Android Wear

Initially a native Android Wear app was developed which utilized the Mes- sage API (2.3.2) to send and receive commands to and from the phone which was connected to the server. The final product however is a lightweight An- droid app which uses enhanced notifications to extend functionality to the smartwatch. This solution was the most logical, from a users perspective, to solve the use case provided. But to get a wider view and more depth of the development process using Android Wear both implementations will be explained along with the implementation of the WebSocket library.

4.3.1 WebSockets For the implementation of the WebSocket protocol in the Android app the client library Java WebSockets (Rajlich (2009)) was used. This was easily done by including the code from Apache Maven (Wikipedia (2015a)) by listing it in the gradle dependencies for the mobile module and importing it to desired file (Listing 4.2). This enables the Android application to connect and communicate with a WebSocket server of our choosing.

Listing 4.2: Include WebSocket library build.gradle (module: mobile) dependencies { compile”org.java −websocket:Java −WebSocket:1.3.0” } MainActivity. java

import org.java websocket. client .WebSocketClient; import org.java websocket .handshake.ServerHandshake;

28 4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION

With this client library included in the project an instance of the ab- stract class WebSocketClient can be instantiated with a WebSocket URI and used to set up the connection and event listeners. The class using WebSocketClient must implement the following methods related to the con- nection to be useful: onOpen, onClose and onMessage. To send a message to it’s connected server the instance can use the send method. The onOpen and onClose events allows us to perform certain tasks when the WebSocket connection opens or closes. This is useful for alerts when disconnected or for example sending a message identifying the device to the server when the connection opens. The actual full implementation of this library in the applications can be found in the Appendices C.3 and D.1.

4.3.2 Native Android Wear As mentioned above initially a native Android Wear application was devel- oped to solve the use case. This means that the applications code is stored, started and runs entirely on the smartwatch device. But due to the nature and limitations of Android Wear explained in the Background section some parts of the code base must be run in the companion app on the phone. An- droid Wear does for instance not have Wi-Fi support as of yet. And since the current use case in question needs to communicate with external systems i.e the web server this had to be handled by the paired phone. This means that the code base for the application is divided into a ’mobile’ module (runs on the phone) and a ’wear’ module (runs on the smartwatch). To be able to communicate the actions performed on the watch to the phone the Message API (2.3.2) was used. This is done by letting your class, in both modules, implement the GoogeApiClient and MessageApi interfaces (Listing 4.3).

Listing 4.3: Implemented interfaces in ’mobile’ module p u b l i c class MediaControlService extends IntentService implements GoogleApiClient.ConnectionCallbacks , MessageApi. MessageListener {

p u b l i c MediaControlService() { super(”MediaControlService”); }

@Override p r o t e c t e d void onHandleIntent(Intent intent) { initGoogleApiClient() ; connectWebSocket() ; } ...

In Listing 4.3 it should be noted that I have chosen to use a IntentService to handle the communication from the phone to the server. This is because IntentService’s in Android can be run in a background thread allowing op-

29 4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION erations to be done even though the app is not open in the foreground. This is crucial to improve the user experience. Both in terms of app responsive- ness but more importantly having the ability to send messages between the devices even though the phone is not running the app in the foreground. In this case the background service MediaControlService is started from the MainActivity directly when the app starts by using the provided service method: startService(mediaControlService). The steps to set up the con- nection, done in onHandleIntent above, is as follows: first do the necessary initialization of the GoogleApiClient, connect and then add the MessageAPI listener (Listing 4.4). This is done for both devices.

Listing 4.4: Initialize GoogleApiClient, connect, add listener p r i v a t e void initGoogleApiClient() { mGoogleApiClient = new GoogleApiClient.Builder(this) .addApi(Wearable .API) . addConnectionCallbacks(this) . b u i l d ( ) ;

i f( mGoogleApiClient != null && !( mGoogleApiClient. isConnected() | | mGoogleApiClient. isConnecting())) mGoogleApiClient. connect() ; }

@Override p u b l i c void onConnected(Bundle bundle) { Wearable.MessageApi. addListener(mGoogleApiClient , this); }

@Override p u b l i c void onMessageReceived(MessageEvent messageEvent) { //Do stuff } ...

Once the connection is established a message API, not very unlike the one described for WebSockets, can be used. The Wearable.MessageApi.addListener( mGoogleApiClient) will register a listener that will be notified when receiv- ing a message. Handling of said message is done in the method onMes- sageReceived (MessageEvent messageEvent) which can perform the intended task depending on the message. Messages are sent asynchronously by ex- tending the AsyncTask class to prevent blocking of the UI thread. Each button in the wear app has a click listener which sends a message to the phone via the MessageAPI. This in turn converts the message to JSON and forwards the command to the server. The server registers that the phone has sent a command and relays it to the VIA application. The message handlers in VIA mentioned above performs the desired command e.g. paus- es/starts video playback. Since the wearable app’s layout does not require any updates depending on interaction done on the phone or VIA no events need to be implemented to handle message receiving. Implementation of

30 4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION the ’wear’ module of this application is shown in Appendix C.2. The im- plementation of the ’phone’ module including MainActivity.java and the background service MediaControlService.java handling WebSockets can be found in Appendix C.3 and Appendix C.4 respectively.

4.3.3 Extended Notification As previously stated an alternate solution to the one explained above was developed to solve the use case. This application utilizes the ability to ex- tend notifications from the phone to the wearable device. This functionality work out of the box with Android Wear and is easily implemented if the existing Android application already supports notifications. By implement- ing the media control in the notification interface the user is able to control the media from both the phone and the wearable, and they are automat- ically in sync. With this solution of extending an interactive notification from the phone the entire code base is written and runs on the phone. The notification is simply extended, customized and displayed on the wearable through the inner workings of Android Wear using WearableExtender. In Android development there is a builder class for creating notification objects (Notification.Builder) and a manager class for managing these (Notification- Manager). When creating the notification a set of parameters is required to specify the attributes of the notification. This includes but is not limited to attributes such as: title, icon, content title, content text, style, actions and extenders. Notification.Builder shown in Listing 4.5.

Listing 4.5: Notification.Builder // Createa WearableExtender to add functionality for wearables Notification .WearableExtender wearableExtender = new Notification .WearableExtender() .setContentAction(1) ;

//Create the notification Notification.Builder notificationBuilder = new Notification.Builder(this) . s e t S t y l e (new Notification.MediaStyle() . setMediaSession(mSession.getSessionToken()) .setShowActionsInCompactView(0 ,1 ,2)) . setVisibility(Notification .VISIBILITY PUBLIC) . setSmallIcon(R.drawable. accedo l o g o ) . setContentTitle(name) .setContentText(description) .addAction(R.drawable. ic action volume down ,””, lowerVolumePending) .addAction( playing ? R.drawable.ic a c t i o n p a u s e : R.drawable.ic a c t i o n p l a y , playing ? getString(R.string.action p a u s e ) : getString(R.string.action p l a y ), playing ? pausePendingIntent

31 4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION

: playPendingIntent) .addAction(R.drawable. ic a c t i o n v o l u m e u p ,””, raiseVolumePending) .setLargeIcon(posterArt) .extend(wearableExtender);

// Get an instance of the NotificationManager service NotificationManager notificationManager = (NotificationManager) getSystemService( NOTIFICATION SERVICE) ;

// Build the notification and issues it with notification manager. notificationManager.notify(notificationId , notificationBuilder. b u i l d ( ) ) ;

The three attributes to make special note of in this case are style, actions and extenders. The setStyle method lets the developer use the pre-defined Notification.MediaStyle which automatically gives us a notification layout style suitable for media control. The ability to add actions to the notification lets the user interact di- rectly on the notification i.e. control the media content of VIA Web. An action consist of a icon, a title and a PendingIntent. Intents in Android are abstract descriptions of operations to be performed. When the user interacts with a action the PendingIntent is triggered which launches the IntentService ActionIntentHandler. The IntenService can handle operations asynchronously and uses the attribute added to the intent with .putEx- tra() in the PendingIntent to identify and broadcast the desired operation back to MainActivity. This in turn sends the corresponding message to the WebSocket server in the same way as previously explained for the native application (Appendices D.1, D.2). As stated before one needs to implement the WearableExtender to cus- tomize and extend the notification to the wearable. This is as simple as creating an instance of Notification.WearableExtender and adding this to the extend method of the notification as shown in Listing 4.5. The setCon- tentAction method sets which notification action shall be available directly on top of the wearable notification. Unlike the native application this solution handle messages received from the web server and update the UI of the notification accordingly. The app reads the command received from the server and changes the play to pause (both icon and action) in the notification, or vice versa depending on the command. This way the Web application and the Android application is in sync and it is obvious to the user if the content is playing or not. To enhance the user experience and appearance of the application the title and poster art of the media that is actually playing have been added to the notification. This is done by using the meta data sent in the JSON message (Listing 4.1) from the server to set the title and retrieve the poster. The poster art is retrieved from the web application through a InputStream

32 4.4. APPLE WATCH CHAPTER 4. IMPLEMENTATION using the provided poster URL and created with a BitmapFactory before it is assigned as the notification icon. A check to see if new content is playing prevents unnecessary requests and retrievals of the poster art. Entire implementation of this solution is available in Appendices D.1 and D.2.

4.4 Apple Watch

What follows is the implementation of the aforementioned use case on the Apple Watch platform. The section is divided into a subsection of the WebSocket library and its implementation along with a subsection with the implementation of the actual WatchKit application.

4.4.1 WebSocket With the design approach of the solution for this use case a WebSocket li- brary is needed for the app to function. There are a variety of client side implementations working with iOS devices, the one I have chosen is called SocketRocket. SocketRocket is a WebSocket client side implementation that is written in Objective-C and works with both iOS and OSX. The easiest way to include this library and get the right dependencies is to use CocoaPods. CocoaPods is a dependency manager with a great deal of libraries avail- able which make including and managing these more simple. CocoaPods is available to install through Ruby and can later add desired dependencies to the project easily by editing a dependency file and installing them in the terminal. Once the SocketRocket dependency is in place the library can be imported in the standard way (Listing 4.6.

Listing 4.6: Import SocketRocket #import

Once this is done the WebSocket connection can be established and the SocketRocket API used to receive and send messages to the web server as shown in Listing 4.7.

Listing 4.7: The SocketRocket API SRWebSocket ∗ webSocket ; //Connect to server − (void)connectWebSocket { i f(webSocket == nil) { webSocket.delegate = nil; webSocket = nil;

NSString ∗ urlString = @”ws://192.168.1.52:1338”;

33 4.4. APPLE WATCH CHAPTER 4. IMPLEMENTATION

SRWebSocket ∗newWebSocket = [[SRWebSocket alloc ] initWithURL :[NSURL URLWithString: urlString ]]; newWebSocket.delegate = self ;

[newWebSocket open ]; } }

#pragma mark − SRWebSocket delegate //webSocketDidOpen − (void )webSocketDidOpen :( SRWebSocket ∗)newWebSocket { webSocket = newWebSocket; [webSocket send :[ NSString stringWithFormat:@”Hello from %@”, [UIDevice currentDevice ].name]]; }

//didFailWithError − (void)webSocket :(SRWebSocket ∗)webSocket didFailWithError :( NSError ∗) e r r o r { [ self connectWebSocket]; }

//didCloseWithCode − (void)webSocket :(SRWebSocket ∗)webSocket didCloseWithCode:( NSInteger)code reason :(NSString ∗)reason wasClean :(BOOL) wasClean { [ self connectWebSocket]; }

//didReceiveMessage − (void)webSocket :(SRWebSocket ∗)webSocket didReceiveMessage :( id ) message { // NSLog(@”Received message:%@”, message);

}

//Senda message [webSocket send:/ ∗ Payload here ∗/];

4.4.2 WatchKit App As we have learned from the theoretical background the entire code base for third party applications for Apple Watch is both stored and run on the companion iPhone. It is located as an extension to the containing iOS appli- cation. The only thing stored on the actual watch is the Storyboard which the phone can send updates to. Since the code already is stored and exe- cuted on the phone only the elements on the Storyboard and the WatchKit extension need to communicate for this solution. The UI elements sends actions to the extension and the extension send updates to the UI. The WebSocket implementation explained above is simply placed in the Inter- faceController.m file and the connection is opened as soon as the user starts the app on the Apple Watch. Interface Builder in Xcode was used during

34 4.4. APPLE WATCH CHAPTER 4. IMPLEMENTATION development to connect the UI controls of Apple Watch to the code in Inter- faceController. IBActions are defined to perform the desired actions when touching a button in the UI. The UI elements representing the play/pause button, title, media description and poster image are defined as outlets. The play/pause buttons background image is changed when the play state of the media is altered. This happens when the user touches the button or a mes- sage is received from the web server that the play state has changed. The title, description and poster image values are set to the meta data extracted from the JSON message (e.g. Listing 4.1 sent from the server when media starts or stops playing in VIA. The class NSJSONSerialization is used to convert the JSON formatted message to a Foundation object or vise versa when communicating with the web server as shown in Listing 4.8 and 4.9.

Listing 4.8: JSON Deseriallization NSDictionary ∗ jsonDictionary ; NSData ∗ jsonData ; jsonData = [message dataUsingEncoding:NSUTF8StringEncoding ]; jsonDictionary = [NSJSONSerialization JSONObjectWithData: jsonData options:0 error:nil ];

Listing 4.9: JSON Seriallization jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @”phonePlay”,@”event”, n i l ] ; NSError ∗ e r r o r = n i l ; jsonData = [NSJSONSerialization dataWithJSONObject: jsonDictionary options :NSJSONWritingPrettyPrinted error :& e r r o r ] ; NSString ∗ jsonString = [[NSString alloc] initWithData: jsonData encoding :NSUTF8StringEncoding ];

[webSocket send: jsonString ];

Since the meta data about the poster image is a URL of the location of the media asset this must be fetched somehow. The URL is extracted from the JSON message but before fetching is the URL of a service in VIA added which blurs images sent to it. This is just because it makes it more aesthetically appealing and makes the text on top more readable, it does not affect the actual implementation. Once the URL is known the data is fetched using NSString stringWithFormat: and the image extracted from the data with UIImage imageWithData:. Due to the layout constraints put on third party developers in WatchKit all the app’s elements had to be put in one group that covers the entire watch screen. This was the only way to set the entire background to the poster image (this is against Apple’s design guidelines). The proccess of fetching the poster and setting the background

35 4.4. APPLE WATCH CHAPTER 4. IMPLEMENTATION image shown in Listing 4.10.

Listing 4.10: Fetch and set poster image NSString ∗ blurString = @”http://blurmaster.apps.accedo.tv/blur?r =8&q=”; NSString ∗command = [jsonDictionary objectForKey:@”event”]; NSString ∗posterURL = [NSString stringWithFormat:@”%@%@”, blurString ,[[ jsonDictionary objectForKey:@”data”] objectForKey :@”poster”]];

NSURL ∗imageURL = [NSURL URLWithString:posterURL ]; NSData ∗imageData = [NSData dataWithContentsOfURL:imageURL]; UIImage ∗ image = [UIImage imageWithData:imageData ]; [[ self mainGroup] setBackgroundImage:image];

The entirety of the WatchKit implementation is found in Appendix E.2 and E.1.

36 Chapter 5

Result

The results and accomplishments during implementation are presented in this chapter. Applications have been developed on the two platforms An- droid Wear and Apple Watch and integrated with a VOD web application, Accedo VIA Web. The purpose of the development of these applications was to discover differences, similarities and get deeper knowledge about the platforms to evaluate cross-platform possibilities. Discussions, analysis and conclusions drawn from the results are reserved for later chapters in the re- port. Only factual results of the development in the form of screen captures from all applications along with a walk-through of the functionalities and features of the apps are presented.

5.1 Android Wear

The result of both of the solutions for the Android Wear platform explained in the Implementation chapter will presented in this section.

5.1.1 Native Android Wear As explained earlier the approach with an native Android Wear application was abandoned fairly early on in benefit of the functionality of the extended notification approach. Thus the interface and look of the application is not the prettiest but all said functions are there. To start a native Android Wear application from the home screen (watch face) the user taps the watch face and is presented with a list of apps/tasks. Third-party applications are located in a sub-menu at the bottom of the list called ’Start...’. By tapping ’Start...’ the user is presented with yet another list containing the available third-party apps installed on the device and can launch them with a single tap on the app of their choosing. Opening the media control app launches the user straight into the the only view in the application, the controls. From here the user can interact with the buttons to PLAY, PAUSE playback

37 5.1. ANDROID WEAR CHAPTER 5. RESULT and raise (+), lower (-) the volume of the media currently playing in the VIA Web application in the web browser. Swiping from left to right when inside the application dismisses and closes it. The final result of the native Android Wear application is presented in Figure 5.1.

Figure 5.1: Final Native Android Wear Application

5.1.2 Extended Notification With the quite tedious series of interactions currently (Android Wear 1.0.5) needed to starting native third-party applications on Android Wear an al- ternate solution was developed. Enabling extended notifications from the phone to the wearable device makes the app more aesthetically pleasing and more inline with how other media control applications function. Addition- ally this adds the functionality to use the control on the phones notification center as well. Launching the companion application on the phone connects to the web server and sets the notification on the phone, and by extension the wearable. Notifications appear in the context stream of the Android Wear operating system which means the user can choose to interact with it, dismiss it or just scroll by it. The notification has been given the highest priority so that is always placed at the top of the context stream and is easily accessible by the user. The high priority and MediaStyle attribute of the notification also enables interaction with the control from the lock screen of both devices. Figure 5.2 show the initial state of the application on both wearable and phone. The phone notification in both compact and expanded state, the Wear notification only in compact view. At this point swiping upward on the wearable opens the expanded view of the notification. This allows the user to get more information about the media, view the poster art and perform additional tasks. Swiping from right to left reveals the volume controls and to the far right the option to

38 5.1. ANDROID WEAR CHAPTER 5. RESULT

Figure 5.2: Home screen with notification on wearable and phone block the app from the smartwatch. There is only one PLAY/PAUSE button which changes state and icon when pressed or if the medias play- state changes in the web application. The expanded view of the notification displaying different play states as well as the lower volume button is shown in Figure 5.3.

Figure 5.3: Final Android Wear Application

39 5.2. APPLE WATCH CHAPTER 5. RESULT

5.2 Apple Watch

When launching the application from the home screen the user is presented with the view which contains the core of the functionality, the media con- trols. Launching the app connects the watch (through the phone) to the go- between web server. The title of the media currently playing is displayed at the top of the watch with the actual media controls positioned underneath. The PLAY/PAUSE button is the same button which changes appearance depending on the play-state of the current media playing. The application uses a page-based navigation style to let the user swipe from right to left to access a view containing a brief description of the media playing. Scrolling of the text can be done by usual user interaction (flicking the finger) or by turning the Digital Crown. Included are screen captures of the application both with and without the poster art set as the background. The Apple design guidelines recommend having a black background at all times to give a more seamless appearance towards the bezel of the watch. The final Apple Watch Application is shown in Figure 5.4.

Figure 5.4: Final Apple Watch Application

Possibilities to implement an app with similar functionality as the no- tification extender for Android exists also for Apple Watch. This was only explored theoretically due to time constraints and the fact of its late release but will be presented nonetheless. Mentioned in subsection WatchKit ar- chitecture and life cycles (2.4.2) WatchKit has the ability to leverage the functionality of some iOS technologies and one of these is the usage of MPRemoteCommandCenter. The Watchkit has the ability to use remote control events of audio and video playing on the companion iPhone through this class. This is done by the so called ”Now Playing” glance. This is a glance (2.4.1) which can send events to the phones command center and dis-

40 5.2. APPLE WATCH CHAPTER 5. RESULT play information about the media from the class MPNowPlayingInfoCenter. This means that an iOS application already utilizing the functionality of MPRemoteCommandCenter and MPNowPlayingInfoCenter could therefore fairly easily be extended to the wrist of the Apple Watch user. Apple illus- trates the Now Playing glance for music playing on the iPhone as shown in Figure 5.5.

Figure 5.5: Now Playing Glance on Apple Watch (Apple, 2015a)

41 Chapter 6

Cross-platform Evaluation

During the implementation phase the APIs of both platforms were explored thoroughly and a explanation of approaches used today and an evaluation of the cross-platform possibilities is presented in this chapter. During the eval- uation the same aspects as when evaluating cross-platform development for smartphones are considered. The approach to evaluation of cross-platform possibilities that is presented in ’A Comparative Analysis of Cross-platform Development Approaches for Mobile Applications’ written by Xanthopoulos and Xinogalos (2013) is used. Considering functionality, access to hardware and APIs on each of the platform for each approach mentioned.

6.1 Approaches

As described in the Theoretical Background Ohrt and Turau (2012) states that there are basically three different techniques of cross-platform devel- opment: mobile web applications, cross-compilation to separate native code and a hybrid approach using the web-view element on each platform. They present that the hybrid approach is right after the mobile web applications the most common in cross-platform development. So the question is: which of these approaches and techniques, if any, are applicable to the smartwatch platforms?

6.1.1 Mobile web application A mobile web application is not a ”app” in the sense that the user installs it on their device from a proprietary app store (e.g. Google Play, App Store). It is rather simply a website which is developed for mobile devices and tries to act and feel like a native app but is not implemented as such. They are typically written using HTML5 and is accessed by the user though the device browser using the URL to the website. The main benefit of an ap- plication of this type is that it is completely platform independent. No real

42 6.1. APPROACHES CHAPTER 6. CROSS-PLATFORM EVAL. knowledge of native smartphone development is needed, only web develop- ment skills. As long as the user has a smartphone with a web browser it can access the application. This approach comes with some limitations. First of all will the mobile web app never have as good responsiveness as a na- tive app (Dhillon and Mahmoud, 2014). The developer cannot access native smartphone APIs and features such as: push notifications, background exe- cution, complex gestures, offline access (except for limited browser caching) and sensor data. The fact that web applications running in the smartphone’s browser cannot access these APIs or send notifications to the phone (which could be extended to the smartwatch) makes this approach to cross-platform development unsustainable in a smartwatch context. There are a few third-party native web browsers developed for Android Wear that can run on the smartwatch even though android.webkit is un- available to developers. This is however not possible in Apple’s WatchKit as explained in 2.4.1 which ultimately renders this solution moot as well.

6.1.2 Hybrid approach Applications that combine the approach of mobile web applications with native elements are called hybrid. Like a native app are hybrid applications available for download and installation from each platforms app store. This solution utilizes each platforms native web view available in the SDKs as a wrapper for the web application’s content. This essentially means that an hybrid application is a mobile web app running inside an native app. There are a variety of tools available using this approach for development of apps e.g. PhoneGap/Cordova and Sencha Touch. The absence of android.webkit in Android Wear and UIWebView in WatchKit as discovered during the pre-study of both platforms makes this approach non viable for applications running/displaying on the smartwatch. However, with hybrid applications being part native gives developers access to native features on the phone such as push notifications. This opens up possibilities thanks to the notification extenders available in both Android Wear and WatchKit as shown during Implementation (4). The handling of notifications in Android and iOS is already available in multiple cross- platform development tools (Inderjeet and Palmieri, 2013) which opens up possibilities. With regular notifications extended right out of the box to smartwatches means that apps developed using this hybrid approach can have some interactivity with a smartwatch.

6.1.3 Cross compilation Native applications are applications that are installed, stored and run on the device. They are developed for a specific platform and take advantage of all native features available in the corresponding specific SDK. This makes na- tive applications superior to the other approaches from a users perspective considering features, responsiveness and look and feel. Cross compilation

43 6.2. POSSIBILITIES CHAPTER 6. CROSS-PLATFORM EVAL. refers to the process of generating native code for multiple platforms from one single code base. Tools using this approach (e.g. Xamarin and MoSync) can take full advantage of the native APIs (as far as they can cross compile) and create native applications ready for deployment to respective application store. Tools of this type are fairly complex compared to development tools utilizing the previously explained approaches. For pure cross-platform de- velopment of native applications (Android Wear) and WatchKit extensions is this the only viable approach because of the reasons explained in 6.1.1 and 6.1.2. In addition to this approach having possibilities to create real cross-platform applications are the same possibilities for notifications avail- able as in the hybrid approach.

6.2 Possibilities

With the three different cross-platform approaches presented we can draw the conclusion that there is no obvious working solution available already today for smartwatches. But this does not mean there are not any possibili- ties to utilize and benefit from these approaches. Both the hybrid and cross compilation approach can partly solve the problem. With these approaches the containing/companion application running on the phone can be for the most part created from a common code base. These in turn have the ability to be hooked up to natively developed components: smartwatch apps/exten- sions for both platforms. So even though the hybrid approach is not a viable approach for creating apps running/displaying on the smartwatch, apps can still have the ability to communicate with them. The PhoneGap/Cordova hybrid development tool for instance have plugins available to send light weight messages/commands to both Android Wear and Apple Watch. How- ever, this requires a plugin for each platform and the app/watch extension themselves still need to be created natively for this solution as well. These plugins utilize the APIs available for communication between phone and smartwatch in each SDK described in chapter 2. This at least opens up for more extensive integration towards smartwatches using a hybrid approach than just forwarding notifications. Generally when developing cross-platform applications are only the busi- ness, cloud and database layers part of the shared code base. The UI is typically developed natively to provide native control patterns and give the app a native look and feel. It is however possible for the UI to be part of the shared code base for mobile app development but this usually results in a worse user experience (Charland and Leroux, 2011). In this case the UI is created programmatically from the cross platform generated source code. This presents a problem in the context of smartwatches. The Android Wear platform has the ability to create UI elements programmatically but WatchKit does not. The entire UI of the Apple Watch can only be built using the interface builder with a storyboard. No UI elements can be created pro- grammatically since the UI is located on the Watch itself (as a storyboard)

44 6.2. POSSIBILITIES CHAPTER 6. CROSS-PLATFORM EVAL. and is limited to only receive updates to these from the WatchKit exten- sion running in the phone. The strict UI design restrictions in WatchKit makes creating a universal app design challenging as well. There is also the architectural difference between the platforms which presents a challenge handling execution. With WatchKit’s limitation that developers are only allowed to work with a watch extension (and not natively on the watch) re- stricts the watch from performing background execution, which is possible in Android Wear.

45 Chapter 7

Discussion

A discussion about the method, implementation, results and cross-platform development is presented in this chapter. The smartwatch became a hot topic last year (2014) when finally starting to reach the consumer’s expecta- tions and receiving backing from almost all major tech manufacturers. The buzz and demand have only grown higher during the process of this thesis work with the release of Apple Watch and rolling updates to Android Wear. A lot has happened in the world of smartwatch development during this short period and the following is some discussion about the method and results presented in this report along with experiences learned.

7.1 Result and Evaluation

In chapter 5 we can see that I have managed to produce applications for both the Android Wear and Apple Watch platforms with the same core functionality. The goal of developing an app with the same core function- ality for both platforms was to discover the different platforms possibilities, limitations and differences. This app tests the design elements as well as communication between devices and to external systems of each platform. This is where lessons learned and discoveries made are presented.

7.1.1 Differences and Similarities Comparing the individual application design guidelines published by both Google and Apple it is clear that both companies has the same approach in mind. Both focus on quick interactions (micro interactions, no longer than 5 seconds), contextual information (right information, at the right time) and data collecting from all available sensors. But they have taken different paths to reach that goal. The UI for handling the contextual information and notifications for instance differ greatly. The UI of Android Wear builds upon a context stream of cards which are stacked vertically. These cards can

46 7.1. RESULT AND EVALUATION CHAPTER 7. DISCUSSION be contextual information delivered from Google Now or regular/actionable notifications (as seen in the app developed for this thesis work) pushed from either phone or watch. Actionable cards have the actions stacked like cards horizontally to the right. Apple Watch’s counterpart to Android Wear’s context stream of cards is Glances. Established in the Theoretical Back- ground (2) are Glances and Notifications in WatchKit, unlike Android Wear, completely separate components. Apple’s UI approach is not as linear as the one seen in Android Wear. Unlike the cards glances are always available to the user anywhere on the watch by swiping upward from the bottom of the screen. Multiple glances are now available in a page-based navigation style. Glance’s customization is compared to Android Wear’s cards very restricted and they cannot contain any actions other than starting the application it represents. Some of the functionalities available in watchOS and the native appli- cations I have learned to be misleading to what actually can be done in WatchKit. The home screen for instance allows for free form gestures, this is not available in WatchKit. Views can only be scrolled vertically and hor- izontal swipes are only used for page navigation. Android Wear has the ability to have a view much larger than the screen which the user can pan around by using free form gestures. The restriction to top-to-bottom UI design in WatchKit is explained in the background and became apparent during implementation. This could be somewhat frustrating and forced cer- tain workarounds (e.g. a Group for the background image) to get the app looking as planned. On the contrary are the possibilities in Android Wear, thanks to the native support, as many (save for limitations listed in 2.3) as when working with Android for smartphones. And with the native support for Android Wear the application is capable of performing background tasks asynchronously as shown in the implementation when sending commands from watch to phone. This prevents the UI thread from being blocked when doing more long running operations, like in this case communicating with a web service. WatchKit’s inability to do this can result in the user seeing a load screen when fetching data. The containing iOS app on the other hand supports background execution. With the watch extension on Apple Watch running entirely on the watch it can communicate directly with the web. So even though this is in fact a pretty big limitation, functionality wise, it facilitates the development greatly. Android Wear on the other hand re- quires the Wear application to have the phone to act as a proxy and relay the messages onward to the web. This, as seen in the implementation, can be a quite complex and time consuming task using the Message and Data API provided in the Android Wear SDK.

Apple’s WatchKit is far more restricted than the Android Wear SDK. This is very much intentional by Apple and can be seen as both negative and positive. With a more restricted sandbox provided to developers there is less room for them to make mistakes. With the finite amount of processing

47 7.1. RESULT AND EVALUATION CHAPTER 7. DISCUSSION power and very limited battery capacity it is crucial to create efficient apps. Even though the development method, architecture, restrictions and UI differs greatly both platforms are more or less capable of bringing the same functionality, in different formats, to the end user.

7.1.2 Challenges and Limitations It can be said that the smartwatch platform in general is a fairly limited one. This was established during the pre-study and then confirmed during implementation. This is partly due to obvious limitation of the screen size available, but also the APIs provided by both platforms. With the lim- ited possibilities of interaction both platforms puts great emphasis on voice interactions. Android Wear has the ability to use Google Now to open appli- cations or dictate messages while Apple Watch supports Siri. But I think it is hard to get the users to use these interaction methods. I do not think the average consumer is comfortable using voice commands and talking to their wrist, especially in a public setting. I think the biggest challenge for even the most seasoned app developers will be to adjust to this new form factor. The concept of micro interactions and keeping the application complexity as low as possible is imperative to produce a high quality smartwatch app. The smartwatches main functions as of today is to primarily act as a notification extender and works very well as this. From a cross-platform perspective are the different paths in UI patterns the two platforms have chosen to take a big challenge to overcome. More on this in section 7.1.3.

7.1.3 Cross-platform Evaluation There are two different ways to consider cross-platform development for smartwatch interaction. The first way is cross-platform development of apps running on the watch (Android Wear)/apps displayed on the watch (WatchKit extension). In the Cross-platform Evaluation (6) is it established that the cross compilation approach is the only viable one to achieve this. This is due to the absence of android.webkit in Android Wear and UIWebView in WatchKit as discovered during the pre-study of both platforms. Cross compilation for smartwatches could be argued to be even more challenging than it is for smartphones. The task of creating a tool capable of this would be challenging because of the large differences of the two platforms. Unlike the case with smartphones are the interaction patterns on each device so dissimilar that an full app would basically need a total UI re-design to function on/be ported from one to the other. That being said I still see possibilities for more simple applications. A smartwatch app which does not utilize the platform specific UI patterns (e.g. cards, glances etc.), and for instance only displays a simple GUI like the native Android Wear app partially developed and presented in this report. A app of that sort could potentially more easily be ported from one platform to the other. If not for the problem of not being able to create UI elements

48 7.2. METHOD CHAPTER 7. DISCUSSION programatically for Apple Watch could this perhaps be done using similar existing (cross compilation) cross-platform methods as for smartphones. The second way of considering cross-platform development for smart- watch interaction is the extension of notifications. Since this is supported without any alterations to an existing application implementing notifications could this be done using both a cross compilation and a hybrid approach. This only applies to regular notifications but with the simplicity of imple- menting smartwatch specific functionality to these I believe abstractions can be made to implement these cross-platform as well. So with the statement that smartwatches today mainly acts as notification extenders, and for noti- fication extension are there some existing cross platform solutions out there.

7.2 Method

Initially, starting this thesis work, I did not know much about either wear- ables or smartwatches. I knew they existed but not really what their capabil- ities and functionality were. With some experience developing smartphone application for both Android and iOS I found this new addition to both platforms and the whole concept of a smartwatch very exciting. The work was carried out in what can be described as a agile method combined with some constantly ongoing tasks. The first two week was spent getting set up at the office and constructing the project plan. The plan con- sisted of two iterative processes done in parallel: the literary research (in- cluding writing this report) and the implementation (actual development). The implementation was conducted iteratively according to the method de- scribed in chapter 3. Similarly the process of writing this report was done iteratively to keep up with the development. This along with the constant ongoing literary research. With a topic this fresh with sparse documenta- tion and few previously written papers the search for relevant sources and references had to be an ongoing process throughout the thesis. This re- sulted in primarily online sources and references due to the lack of printed literature related to smartwatches and all platform specific information and manuals being online. The initial idea (and implementation) was to make simple HTTP requests to the go-between server to handle the media con- trol commands. When the discovery and idea of using WebSockets instead emerged the server was restructured and WebSocket libraries was added to each platform. This change in the solution did not take too much time and resulted in a better finalized application. The decision to abandon the native approach for Android Wear and explore the functionality of extending notifications was initially taken in benefit to solve the use case at hand. But in the end this proved valuable from an cross-platform perspective as well, giving insights to the possibilities of notifications on the smartwatch. A constant challenge and limitation during the entirety of the thesis work was the immaturity of both platforms, most noticeably Apple Watch. With

49 7.2. METHOD CHAPTER 7. DISCUSSION

WatchKit, initially, in early beta and new functionality being released to Android Wear sporadically a lot of time was spent reading on developers blogs and forums. The NowPlayingGlance presented in 5.2 arrived in Beta 5 (February) for instance. The full version of WatchKit itself, albeit very limited for third-party developers, was released in March. In addition to this was the absents of a physical Apple Watch device frustrating. It was very limiting not being able to run the application on a real device and explore how the watch behaved. Time spent on each smartwatch platform during development could have been planned better, but this worked out fairly well in the end, much thanks to the fluidity of agile method applied. Deciding to deep dive into two different platforms could be argued to have been too much. During the work I managed to get a good grasp of both platforms APIs, differences and capabilities. But I would have liked to have more generous time constraints to really become a expert on both platforms.

50 Chapter 8

Conclusion

Within this thesis the two smartwatch platforms Android Wear and Apple’s WatchKit have been investigated in the purpose of finding similarities, dif- ferences and to evaluate cross-platform development possibilities. Based on the results, evaluation and discussion presented earlier in this report some conclusions could be made. It was found that the platforms differ quite a bit regarding architecture and approach to UI even though they build upon the same principles. Even though no existing comprehensive solution for pure cross-platform development for smartwatches was found during this thesis are there some possibilities as presented and discussed in chapter 6 and 7. The smartwatch platform is a fairly new and immature one which is evolving rapidly. With rolling updates to both platforms and with native capabilities coming to WatchKit in the near future is it important to reevaluate the results of this thesis with the newest possible versions of both SDKs. The research questions presented in the introduction and focus for this thesis were: • Which are the integration points with the wearable devices APIs and external systems1, which can maximize portability and development synergies cross-platform? • Which are the similarities and differences between the smartwatch platforms from both a developer’s and a user’s perspective?

With the Theoretical Background and Implementation together with the Cross-platform Evaluation and Discussion I believe these questions were answered in a satisfactory fashion. That being said, I believe that there is always room for improvements and that there is also room to do further work on this investigation of the platforms APIs, especially after significant updates gets released.

1Outside the smartwatch e.g. companion phone or web.

51 8.1. FUTURE WORK CHAPTER 8. CONCLUSION

8.1 Future work

First of all will the release and update to WatchKit that lets developers get access to the entire Apple Watch and develop native applications change the playing field quite a bit. Both Android Wear and WatchKit are constantly evolving and updates are released continuously. In such a new, growing and immature area are there a lot of changes taking place all the time from many different large players. New hardware features gets released to Android Wear and a new version of the Apple Watch is to be expected in the coming years. This opens up for more functionality and more freedom when using the smartwatch platform. During this thesis has a new version of Android Wear which enables certain watches to use Wi-Fi for internet connectivity been released (thereby avoiding the dependency of the phone as seen in the implementation (4)). It is imperative to continuously keep one self updated with both investigated platforms (changes to the APIs, new features etc.) to discover new possibilities of cross-platform development. Since starting this thesis have signs of even more fragmentation in the smartwatch market been showing. Even though both LG and Samsung are announced as partners to Google and Android Wear have both manufac- turers gone the own way as well. LG recently released the smartwatch LG Watch Urbane LTE which, as the name suggests, has LTE support. This watch however runs LG’s own WebOS. Meanwhile is Samsung still pushing for their own operating system Tizen and running this on several wearable devices. Only time will tell where and when the smartwatch platforms set- tles.

52 Bibliography

Accedo. Accedo. http://www.accedo.tv/company/, 2015a. [Online; ac- cessed 01-April-2015]. Accedo. Accedo via. http://www.accedo.tv/solutions/accedo-via/, 2015b. [Online; accessed 01-April-2015]. Chloe Albanesius. Smartwatches has thier time come? PC Magazine, June: 99–106, 2014. Apple. Developers start designing apps for apple watch. Apple Press Info, November 2014. URL http://www.apple.com/pr/library/2014/11/ 18Developers-Start-Designing-Apps-for-Apple-Watch.html. [On- line; accessed 11-May-2015]. Apple. Now playing glance apple watch. www.apple.com/watch, 2015a. [Online; accessed 27-April-2015]. Apple. Watchkit app communication. https://developer.apple. com/library/prerelease/ios/documentation/General/Conceptual/ WatchKitProgrammingGuide/DesigningaWatchKitApp.html, 2015b. [Online; accessed 9-March-2015]. Apple. Watchkit app life cycle. https://developer.apple.com/ library/prerelease/ios/documentation/General/Conceptual/ WatchKitProgrammingGuide/DesigningaWatchKitApp.html, 2015c. [Online; accessed 9-March-2015]. Charles Arthur. Android is more fragmented than ever. should developers or users worry? The Guardian, August 2014. http://www.theguardian.com/technology/2014/aug/22/android- fragmented-developers-opensignal. Andre Charland and Brian Leroux. mobile application Development : Web vs . native. Communications of the ACM, 54:0–5, 2011. ISSN 00010782. doi: 10.1145/1941487. Sunny Dhillon and Qusay H. Mahmoud. An evaluation framework for cross- platform mobile application development tool, 2014.

53 BIBLIOGRAPHY BIBLIOGRAPHY

Vincenzo Ferrari. ng-websocket. https://github.com/wilk/ng- websocket, 2014. [Online; accessed 25-February-2015]. Google. Google play services diagram. https://developer.android. com/google/play-services/index.html, 2015. [Online; accessed 28- February-2015]. Google’s. Android wear. http://developer.android.com/wear/index. html, 2015. [Online; accessed 01-April-2015]. Michael Hahn. Data layer stack. http://android-wear-docs. readthedocs.org/en/latest/sync.html, 2015. [Online; accessed 11- March-2015]. Singh Inderjeet and Manuel Palmieri. Comparison of Cross-Platform Mobile Development Tools, M¨alardalenUniversity Innovation , Development and Technology. 2013. Timothy Jordan. An introduction to android wear. https://www.youtube. com/watch?v=Bl4Qne-RpcM, 2014. [Online; accessed 30-January-2015]. Inc. Kickstarter. Kickstarters most funded campaigns. https://www. kickstarter.com/discover/most-funded, 2015. [Online; accessed 01- April-2015]. Victor Lipman. 71to wear a watch? Forbes, September 2014. http: //www.forbes.com/sites/victorlipman/2014/09/22/71-of-16-24s- want-wearable-tech-why-dont-i-even-want-to-wear-a-watch/. Brian McKelvey. Websocket-node. https://github.com/theturtle32/ WebSocket-Node, 2015. [Online; accessed 25-February-2015]. NextMarket. Smartwatch forecast 2013-2020 (single report). http://nextmarket.co/blogs/news-1/9278081-smartwatch-market- forecast-to-grow-from-15-million-in-2014-to-373-million-by- 2020, 2013. [Online; accessed 02-February-2015]. Julian Ohrt and Volker Turau. Cross-platform development tools for smart- phone applications. Computer, 45:72–79, 2012. ISSN 00189162. doi: 10.1109/MC.2012.121. Dipesh Pradhan and Nugroho Sujatmiko. Can smartwatch help users save time by making processes efficient and easier. Final report, Universitetet i Oslo, November 2014. Nathan Rajlich. Java websocket. https://github.com/TooTallNate/ Java-WebSocket, 2009. [Online; accessed 9-April-2015]. Reza Rawassizadeh, Blaine A. Price, and Marian Petre. Wearables: Has the age of smartwatches finnaly arrived? Communications of the ACM, 58 (1):45–47, January 2015.

54 BIBLIOGRAPHY BIBLIOGRAPHY

Vinod Seettharamu Keshav, Joy Bose, Sunkara Sowmaya, and Nitesh Tigga. Tv remote control via wearable smart watch device. Technical report, Samsung R&D Institute, 2014.

Chris Sevilleja. Mean stack. https://scotch.io/tutorials/creating- a-single-page-todo-app-with-node-and-angular, 2015. [Online; ac- cessed 12-March-2015]. Tractica. Smart watch shipments will more than quadruple in 2015, reaching 24.4 million units worldwide. https://www.tractica.com/newsroom/ press-releases/smart-watch-shipments-will-more-than- quadruple-in-2015-reaching-24-4-million-units-worldwide, 2015. [Online; accessed 02-February-2015].

Wikipedia. Apache maven. http://en.wikipedia.org/wiki/Apache_ Maven, 2015a. [Online; accessed 9-April-2015].

Wikipedia. Smartwatches. http://en.wikipedia.org/wiki/Smartwatch# Early_years, 2015b. [Online; accessed 28-January-2015]. Spyros Xanthopoulos and Stelios Xinogalos. A Comparative Analysis of Cross-platform Development Approaches for Mobile Applications. Pro- ceedings of the 6th Balkan Conference in Informatics, pages 213–220, 2013. doi: 10.1145/2490257.2490292.

55 Appendix A

Web server

Listing A.1: server.js, Requiring and starting the WebSocket server var http = require(’http’); var url = require(’url’); var webSocketServer = require(’./webSocketServer’); function start(route , handler) { var httpServer = http.createServer(function (req, res) { var pathname = url.parse(req.url).pathname;

route(pathname, handler , res);

}).listen(1338,’0.0.0.0’);

console.log(’Server running at http://127.0.0.1:1338/’); webSocketServer. start(httpServer); } exports.start = start;

Listing A.2: webSocketServer.js, WebSocket server implementation var server = require(’./server’); var WebSocketServer = require(’websocket’).server; var connection; var connections = {} ; var connection c o u n t e r = 0 ; var isPlaying; var messageData; // Create an object for the websocket function start(httpServer) { wsServer = new WebSocketServer({ httpServer: httpServer , autoAcceptConnections : false

56 APPENDIX A. WEB SERVER

});

// Createa callback to handle each connection request wsServer.on(’request’, function(request) { connection = request.accept(’’);

console.log((new Date()) +’ Connection accepted from ’ + request.origin);

// Callback to handle each message from the client connection .on(’message’, function(message) { i f (message.type ===’utf8’) {

console.log(’Received Message: ’ + message. utf8Data ) ; t r y { messageData = JSON. parse(message.utf8Data); switch(messageData. event) { c a s e”phone”: connections[0] = connection; i f(isPlaying) { sendCommandToPhone(message . utf8Data) ; } e l s e { sendCommandToPhone(message . utf8Data) ; } break; c a s e”browser”: connections[1] = connection; break; c a s e”webPause”: sendCommandToPhone( message . utf8Data ) ; i s P l a y i n g = false; break; c a s e”webPlay”: sendCommandToPhone( message . utf8Data ) ; i s P l a y i n g = true; break; c a s e”phonePause”: sendCommandToBrowser(’pause’); break; c a s e”phonePlay”: sendCommandToBrowser(’play’); break; c a s e”phoneVolUp”: sendCommandToBrowser(’volUp’); break; c a s e”phoneVolDown”: sendCommandToBrowser(’volDown’); break; d e f a u l t: console.log(”No command executed ”); } }

57 APPENDIX A. WEB SERVER

catch (e) { console.log(”Non−JSON message”); } } e l s e if (message.type ===’binary’) { console.log(’Received Binary Message o f ’+ message.binaryData.length +’ bytes’); } });

// Callback when client closes the connection connection .on(’close’ , function(reasonCode , description) { console.log((new Date()) +’ Peer ’+ connection.remoteAddress +’ disconnected.’); }); }); } function sendCommandToBrowser(command) { i f(connections[1] != null) { console.log(”Sending message to browser: ” + command) ; connections [1].sendUTF(command) ; } e l s e { console.log(”WebSocket to browser i s not open yet”); } } function sendCommandToPhone(command) { i f(connections[0] != null) { console.log(”Sending message to phone: ” + command) ; connections [0].sendUTF(command) ; } e l s e { console.log(”WebSocket to phone i s not open yet”); } } exports.start = start; exports .sendCommandToBrowser = sendCommandToBrowser; exports .sendCommandToPhone = sendCommandToPhone;

58 Appendix B

VIA Web application

Listing B.1: index.html, Relevant dependencies in index.html

... ...

... ...

Listing B.2: webSocketService.js, WebSocket service implementation /∗∗ ∗ Created by gustavnoren on 19/02/15. ∗/ ’use s t r i c t’; angular .module(’webSocket’,[’ngWebsocket’]) . s e r v i c e (’webSocketService’, function ($websocket, $timeout) {

var ws = $websocket.$new({ u r l :’ws://127.0.0.1:1338’, l a z y : true, protocols: []

});

var shouldListen = true; var callback;

59 APPENDIX B. VIA WEB APPLICATION

function sendPause(metaData) { ws . $emit (’webPause’ , metaData); }

function sendPlay(metaData) { ws . $emit (’webPlay’ , metaData); }

function registerListener( c a l l b a c k ) { c a l l b a c k = c a l l b a c k ; }

ws . $on (’$open’, function () { console.log(’The ngWebsocket has open!’);// It will p r i n t after5(or more) seconds ws . $emit (’browser’); });

ws . $on (’$message’, function($message) { console.log(’Message: ’ + $message );

callback($message);

});

$timeout(function () { ws.$open() ;// Open the connction only at this point. It w i l l fire the’$open’ event } , 5000) ;

r e t u r n { sendPause: sendPause , sendPlay: sendPlay , registerListener: registerListener , shouldListen: shouldListen } ; });

Listing B.3: videoplayer.js // // { //”description”:[”The videoplayer directive contains the functionality for the reusable video player.”, //”The required attributes are:’url’(video url),’backdrop ’(image which will be blurred and used as backdrop),’text’ (to show along with the play button),’startFrom’(start p o s i t i o n).”, //”The attributes’onPlay’ and’onPause’ will be called when the user plays or pauses the playback and can be used by the controller to perform custom logic, such as add the media to the user’s history.”] // } //

60 APPENDIX B. VIA WEB APPLICATION

’use s t r i c t’; angular .module(’tve. videoplayer’,[’tve.play −button’,’webSocket ’]) .directive(’tveVideoPlayer’,[’$sce’,’tveConstants’,’ webSocketService’, function($sce , tveConstants , webSocketService) {

r e t u r n { r e s t r i c t :’E’, r e p l a c e : true, scope : { ’url’:’=’, ’backdrop’:’=’, ’text’:’=’, ’onPause’:’&’, ’onPlay’:’&’, ’startFrom’:’=’ } , templateUrl :’/partials/directives/videoPlayer’, controller: [’$scope’, function($scope) {

var assetHaveBeenPlayed = false; var unregisterStartFromWatcher;

var metaData = { ’poster’: $scope.backdrop, ’title’: $scope.$parent.movie.title , ’description’: $scope.$parent.movie.description } ;

console.log($scope);

webSocketService. registerListener(function(message) { //console.log(message); webSocketService.shouldListen = false; i f (message ===’play’) { $scope.API.play() ; } e l s e if (message ===’pause’) { $scope.API.pause() ; } e l s e if (message ===’volUp’) { $scope .API.volume = $scope .API.mediaElement [0].volume; console.log(’Volume: ’ + $scope.API.volume); i f ($scope .API.volume <= 0 . 9 ) { $scope .API.setVolume($scope .API.volume + 0 . 1 ) ; } } e l s e if (message ===’volDown’) { $scope .API.volume = $scope .API.mediaElement [0].volume; console.log(’Volume: ’ + $scope.API.volume); i f ($scope .API.volume >= 0 . 1 ) { $scope .API.setVolume($scope .API.volume −

61 APPENDIX B. VIA WEB APPLICATION

0 . 1 ) ; } } webSocketService.shouldListen = true; }); function setStartFrom() { unregisterStartFromWatcher = $scope.$watch(’ startFrom’, function (newValue) {

i f (newValue) { $scope .API.seekTime(newValue) ; } }); }

$scope.isPaused = true; // $scope.config= {} ; // $scope.config.theme=’bower components/ v i d e o g u l a r −themes−d e f a u l t/videogular.css’;

$scope.onPlayerReady = function onPlayerReady(API) { $scope.API = API; $scope .API.mediaElement. bind(’loadedmetadata’, setStartFrom); } ;

$scope.onUpdateState = function onUpdateState(state) { i f (state ===’play’ && $scope.onPlay && webSocketService. shouldListen) { $scope.isPaused = false; assetHaveBeenPlayed = true; $scope .onPlay({ time: $scope.API.currentTime /1000}); webSocketService . sendPlay(metaData) ; } e l s e if (state ===’pause’ && $scope.onPause && webSocketService. shouldListen) { $scope .onPause({ time: $scope.API.currentTime /1000}); webSocketService .sendPause(metaData) ; } } ;

$scope.onClickScreen = function onClickScreen() { $scope.API.pause() ; $scope.isPaused = true; console.log(”Backdrop: ” + $scope.backdrop); } ;

$scope.$on(’$destroy’, function() { i f ($scope.onPause && assetHaveBeenPlayed) { $scope .onPause({ time: $scope.API.currentTime /1000}); } i f (unregisterStartFromWatcher) { unregisterStartFromWatcher() ;

62 APPENDIX B. VIA WEB APPLICATION

} }); } ], link: function(scope) { scope.config = { s o u r c e s : [ { src: $sce.trustAsResourceUrl(scope.url), type :’video/mp4’ } ], t r a c k s : [ ] , p l u g i n s : { poster: tveConstants.blurmaster +’/blur?r =8&q=’ + scope.backdrop } } ; } } ; } ]);

63 64 APPENDIX C. NATIVE ANDROID WEAR

Appendix C

Native Android Wear

Listing C.1: Java WebSocket API p r i v a t e void connectWebSocket() { URI u r i ; t r y { u r i = new URI(”ws://192.168.1.52:1338/”); } catch (URISyntaxException e) { e.printStackTrace() ; r e t u r n; }

mWebSocketClient = new WebSocketClient(uri) { @Override p u b l i c void onOpen(ServerHandshake serverHandshake) { Log . i (”Websocket”,”Opened”); isConnected = true; }

@Override p u b l i c void onMessage(String s) { //Perform commands corresponding to the message. }

@Override p u b l i c void onClose(int i, String s, boolean b) { Log . i (”Websocket”,”Closed ”+ s); isConnected = false; }

@Override p u b l i c void onError(Exception e) { Log . i (”Websocket”,”Error ” + e.getMessage()); isConnected = false; } } ; mWebSocketClient. connect() ; }

65 APPENDIX C. NATIVE ANDROID WEAR

Listing C.2: MainActivity.java, ’wear’ module implementation on Android Wear 1 2 public class MainActivity extends Activity implements GoogleApiClient.ConnectionCallbacks , MessageApi. MessageListener { 3 4 private Button playButton; 5 private Button pauseButton; 6 private Button volumeUp; 7 private Button volumeDown; 8 9 GoogleApiClient mGoogleApiClient; 10 public static final String PLAY PATH =”phonePlay”; 11 public static final S t r i n g PAUSE PATH =”phonePause”; 12 public static final S t r i n g VOLUME UP PATH =”volumeup”; 13 public static final S t r i n g VOLUME DOWN PATH =”volumedown”; 14 public static final String TAG =”Debug... MainActivity Wear” ; 15 16 @Override 17 protected void onCreate(Bundle savedInstanceState) { 18 super.onCreate(savedInstanceState); 19 setContentView(R.layout.activity m a i n ) ; 20 21 mGoogleApiClient = new GoogleApiClient.Builder(this) 22 .addApi(Wearable.API) 23 .addConnectionCallbacks(this) 24 . b u i l d ( ) ; 25 26 final WatchViewStub stub = (WatchViewStub) findViewById( R. i d . watch view stub ) ; 27 stub.setOnLayoutInflatedListener(new WatchViewStub. OnLayoutInflatedListener() { 28 @Override 29 public void onLayoutInflated(WatchViewStub stub) { 30 setClickListeners(stub); 31 } 32 }); 33 34 mGoogleApiClient.connect(); 35 } 36 37 @Override 38 public void onConnected(Bundle bundle) {} 39 40 private void sendStartActivityMessage(String path, String nodeId ) { 41 Wearable.MessageApi.sendMessage( 42 mGoogleApiClient, nodeId, path, new byte[0]). setResultCallback( 43 new ResultCallback () { 44 @Override 45 public void onResult(MessageApi. SendMessageResult sendMessageResult) {

66 APPENDIX C. NATIVE ANDROID WEAR

46 if (!sendMessageResult.getStatus(). isSuccess()) { 47 Log . e (TAG,”Failed to send message with s t a t u s code: ” 48 +sendMessageResult. getStatus(). getStatusCode()); 49 System.out.println(”In sendStartActivityMessage − FAIL” ); 50 } 51 else 52 System.out.println(”Message s e n t successfully with s t a t u s code: ” 53 +sendMessageResult. getStatus(). getStatusCode()); 54 } 55 } 56 ) ; 57 } 58 59 private Collection getNodes ( ) { 60 HashSet r e s u l t s = new HashSet <>(); 61 NodeApi.GetConnectedNodesResult nodes = 62 Wearable.NodeApi.getConnectedNodes( mGoogleApiClient) .await() ; 63 for (Node node : nodes.getNodes()) { 64 results.add(node.getId()); 65 } 66 return results; 67 } 68 69 private class SendAsyncMessage extends AsyncTask { 70 @Override 71 protected Void doInBackground(String ... paths) { 72 73 C o l l e c t i o n nodes = getNodes(); 74 System.out.println(”Number on connected nodes: ”+ nodes.size()); 75 for (String path: paths) { 76 for (String node : nodes) { 77 sendStartActivityMessage(path, node); 78 } 79 } 80 return null; 81 } 82 83 protected void onProgressUpdate(Integer ... progress) { 84 } 85 86 protected void onPostExecute(Long result) { 87 } 88 } 89 90 @Override

67 APPENDIX C. NATIVE ANDROID WEAR

91 public void onMessageReceived(MessageEvent messageEvent) { 92 93 } 94 95 @Override 96 public void onConnectionSuspended(int i) { 97 98 } 99 100 private void setClickListeners(WatchViewStub stub) { 101 playButton = (Button) stub.findViewById(R.id.playButton) ; 102 playButton.setOnClickListener(new View.OnClickListener() { 103 public void onClick(View v) { 104 new SendAsyncMessage() . execute(PLAY PATH) ; 105 } 106 }); 107 pauseButton = (Button) stub.findViewById(R.id. pauseButton) ; 108 pauseButton.setOnClickListener(new View.OnClickListener () { 109 public void onClick(View v) { 110 new SendAsyncMessage() . execute(PAUSE PATH) ; 111 } 112 }); 113 volumeUp = (Button) stub.findViewById(R.id.volumeUp); 114 volumeUp.setOnClickListener(new View.OnClickListener() { 115 public void onClick(View v) { 116 new SendAsyncMessage() . execute(VOLUME UP PATH) ; 117 } 118 }); 119 volumeDown = (Button) stub.findViewById(R.id.volumeDown) ; 120 volumeDown.setOnClickListener(new View.OnClickListener() { 121 public void onClick(View v) { 122 new SendAsyncMessage() . execute(VOLUME DOWN PATH) ; 123 } 124 }); 125 } 126 }

Listing C.3: MainActivity.java, ’phone’ module implementation on Android package com.example.gustavnoren.androidmaize;

import android.content.Intent; import android.support.v7.app.ActionBarActivity; import android.os.Bundle; import android.view.Menu; import android.view.MenuItem;

68 APPENDIX C. NATIVE ANDROID WEAR

p u b l i c class MainActivity extends ActionBarActivity { p u b l i c static final String TAG =”Debug... MainActivity Mobile”; p r i v a t e Intent mediaControlService;

@Override p r o t e c t e d void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R. layout . activity m a i n ) ;

mediaControlService = new Intent(this, MediaControlService .class); startService(mediaControlService); }

@Override p u b l i c boolean onCreateOptionsMenu(Menu menu) { // Inflate the menu; this adds items to the action bar i f it is present. getMenuInflater() . inflate (R.menu.menu main, menu); r e t u r n true; }

@Override p u b l i c boolean onOptionsItemSelected(MenuItem item) { // Handle action bar item clicks here. The action bar w i l l // automatically handle clicks on the Home/Up button, so long // as you specifya parent activity in AndroidManifest. xml. i n t id = item.getItemId();

//noinspection SimplifiableIfStatement i f (id ==R.id.action s e t t i n g s ) { r e t u r n true; }

r e t u r n super.onOptionsItemSelected(item); } }

Listing C.4: MediaControlService.java, ’phone’ module implementation on Android package com.example.gustavnoren.androidmaize; import android.app.IntentService; import android.content.Intent; import android.os.AsyncTask; import android.os.Build; import android.os.Bundle; import android.util.Log;

69 APPENDIX C. NATIVE ANDROID WEAR

import com. google .android.gms.common.api.GoogleApiClient; import com. google .android.gms.wearable.MessageApi; import com. google .android.gms.wearable.MessageEvent; import com.google.android.gms.wearable.Wearable; import org.java websocket. client .WebSocketClient; import org.java websocket .handshake.ServerHandshake; import org.json.JSONException; import org.json.JSONObject;

/∗∗ ∗ Created by gustavnoren on 04/02/15. ∗/ p u b l i c class MediaControlService extends IntentService implements MessageApi.MessageListener , GoogleApiClient. ConnectionCallbacks { p u b l i c static final String TAG =”Debug... MediaControl”;

GoogleApiClient mGoogleApiClient; WebSocketClient mWebSocketClient; p r i v a t e boolean isConnected;

p u b l i c MediaControlService() { super(”MediaControlService”); }

@Override p r o t e c t e d void onHandleIntent(Intent intent) { initGoogleApiClient() ; connectWebSocket() ; Log . d (TAG,”onHandleIntent ”); }

p r i v a t e void initGoogleApiClient() { mGoogleApiClient = new GoogleApiClient.Builder(this) .addApi(Wearable .API) . addConnectionCallbacks(this) . b u i l d ( ) ;

i f( mGoogleApiClient != null && !( mGoogleApiClient. isConnected() | | mGoogleApiClient. isConnecting())) mGoogleApiClient. connect() ; }

@Override p u b l i c void onMessageReceived(MessageEvent messageEvent) { //Do stuff Log.d(TAG, messageEvent.getPath()) ;

JSONObject jsonCommand = new JSONObject() ;

i f (messageEvent.getPath() != null && isConnected) { t r y { jsonCommand. put(”event”, messageEvent.getPath() . toString()); } catch (JSONException e) {

70 APPENDIX C. NATIVE ANDROID WEAR

e.printStackTrace() ; }

System.out. println(”jsonCommand: ” + jsonCommand. toString());

mWebSocketClient. send(jsonCommand. toString () ) ; } }

@Override p u b l i c void onConnected(Bundle bundle) { Wearable.MessageApi. addListener(mGoogleApiClient , this); }

@Override p u b l i c void onConnectionSuspended(int i) {

} p r i v a t e void connectWebSocket() { URI u r i ; t r y { u r i = new URI(”ws://192.168.1.52:1338/”); } catch (URISyntaxException e) { e.printStackTrace() ; r e t u r n; }

mWebSocketClient = new WebSocketClient(uri) { @Override p u b l i c void onOpen(ServerHandshake serverHandshake) { Log . i (”Websocket”,”Opened”); isConnected = true; JSONObject jsonCommand = new JSONObject() ; mWebSocketClient. send(”Hello from ” + Build. MANUFACTURER +” ” + Build .MODEL) ; t r y { jsonCommand. put(”event”,”phone”); } catch (JSONException e) { e.printStackTrace() ; } mWebSocketClient. send(jsonCommand. toString () ) ; }

@Override p u b l i c void onMessage(String s) { //Do stuff }

@Override p u b l i c void onClose(int i, String s, boolean b) { Log . i (”Websocket”,”Closed ”+ s); isConnected = false; }

71 APPENDIX C. NATIVE ANDROID WEAR

@Override p u b l i c void onError(Exception e) { Log . i (”Websocket”,”Error ” + e.getMessage()); isConnected = false; } } ; mWebSocketClient. connect() ; } }

72 Appendix D

Android Wear Notification extender

Listing D.1: MainActivity.java, ’phone’ module implementation on Android package com.example.gustavnoren.androidmaize; import android.annotation.TargetApi; import android.app.Activity; import android.app.Notification; import android.app.NotificationManager; import android.app.PendingIntent; import android.content.BroadcastReceiver; import android.content.Context; import android.content.Intent; import android.content.IntentFilter; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.media. session .MediaController; import android.media. session .MediaSession; import android.media. session .MediaSessionManager; import android.media. session .PlaybackState; import android.os.AsyncTask; import android.os.Build; import android.os.Bundle; import android.util.Log; import android.view.Menu; import android.view.MenuItem; import android.support.v4.app.NotificationManagerCompat; import org.java websocket. client .WebSocketClient; import org.java websocket .handshake.ServerHandshake; import org.json.JSONException; import org.json.JSONObject; import java.io.InputStream;

73 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

import java .net.MalformedURLException; import java.net.URI; import java.net.URISyntaxException; import java.net.URL; p u b l i c class MainActivity extends Activity { p u b l i c static final String TAG =”MainActivity Mobile”;

p r i v a t e Intent actionHandlerService; p u b l i c boolean playing = false; WebSocketClient mWebSocketClient; p r i v a t e boolean isConnected;

p r i v a t e MediaSession mSession; p r i v a t e MediaSession.Token mSessionToken; p r i v a t e MediaController mController; p r i v a t e MediaSessionManager mSessionManager;

p r i v a t e PlaybackState mPlaybackState;

p r i v a t e ResponseReceiver responseReceiver;

p r i v a t e URL posterURL; p r i v a t e Bitmap posterArt; p r i v a t e String title =””;

@TargetApi( Build .VERSION CODES. LOLLIPOP) @Override p r o t e c t e d void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R. layout . activity m a i n ) ;

mSessionManager = (MediaSessionManager) getSystemService (Context .MEDIA SESSION SERVICE) ; mSession = new MediaSession(getApplicationContext() ,” mediaSession”); mController = new MediaController(getApplicationContext () , mSession.getSessionToken());

isConnected = false; connectWebSocket() ;

setNotification(”Accedo VIA Web”,”Accedo VIA”);

actionHandlerService = new Intent(this, ActionIntentHandler .class); startService(actionHandlerService);

IntentFilter filter = new IntentFilter(ResponseReceiver. ACTION RESP) ; filter . addCategory(Intent .CATEGORY DEFAULT) ; responseReceiver = new ResponseReceiver(); registerReceiver(responseReceiver , filter);

}

74 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

@TargetApi( Build .VERSION CODES. LOLLIPOP) p r i v a t e void setNotification(String name, String description ) { i n t notificationId = 001;

mPlaybackState = mController.getPlaybackState() ;

/∗∗∗ ∗ I n t e n t s to handle actions done on the notification ∗/ Intent playIntent = new Intent(this , ActionIntentHandler .class); playIntent .putExtra(”command”,”play”); PendingIntent playPendingIntent = PendingIntent. getService(this, 0, playIntent, PendingIntent .FLAG UPDATE CURRENT) ;

Intent pauseIntent = new Intent(this, ActionIntentHandler .class); pauseIntent .putExtra(”command”,”pause”); PendingIntent pausePendingIntent = PendingIntent. getService(this , 1, pauseIntent, PendingIntent .FLAG UPDATE CURRENT) ;

Intent raiseVolumeIntent = new Intent(this, ActionIntentHandler .class); raiseVolumeIntent .putExtra(”command”,”volUp”); PendingIntent raiseVolumePending = PendingIntent. getService(this, 2, raiseVolumeIntent , PendingIntent. FLAG UPDATE CURRENT) ;

Intent lowerVolumeIntent = new Intent(this, ActionIntentHandler .class); lowerVolumeIntent .putExtra(”command”,”volDown”); PendingIntent lowerVolumePending = PendingIntent. getService(this, 3, lowerVolumeIntent , PendingIntent. FLAG UPDATE CURRENT) ;

// Createa WearableExtender to add functionality for wearables Notification .WearableExtender wearableExtender = new Notification .WearableExtender() .setContentAction(1) ;

Notification.Builder notificationBuilder = new Notification.Builder(this) . s e t S t y l e (new Notification.MediaStyle() . setMediaSession(mSession. getSessionToken()) . setShowActionsInCompactView ( 0 , 1 , 2 ) ) . setVisibility(Notification. VISIBILITY PUBLIC) . setSmallIcon(R.drawable. accedo l o g o )

75 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

. setContentTitle(name) .setContentText(description) .addAction(R. drawable. ic action volume down ,””, lowerVolumePending) .addAction( playing ? R.drawable. i c a c t i o n p a u s e : R.drawable. i c a c t i o n p l a y , playing ? getString(R.string. a c t i o n p a u s e ) : getString(R.string. a c t i o n p l a y ) , playing ? pausePendingIntent : playPendingIntent) .addAction(R. drawable. i c a c t i o n v o l u m e u p ,””, raiseVolumePending) .setLargeIcon(posterArt) .extend(wearableExtender);

// Get an instance of the NotificationManager service NotificationManager notificationManager = (NotificationManager) getSystemService( NOTIFICATION SERVICE) ;

// Build the notification and issues it with notification manager. notificationManager.notify(notificationId , notificationBuilder.build()); }

@Override p u b l i c boolean onCreateOptionsMenu(Menu menu) { // Inflate the menu; this adds items to the action bar i f it is present. getMenuInflater() . inflate (R.menu.menu main, menu); r e t u r n true; }

@Override p u b l i c boolean onOptionsItemSelected(MenuItem item) {

// Handle action bar item clicks here. The action bar w i l l // automatically handle clicks on the Home/Up button, so long // as you specifya parent activity in AndroidManifest. xml. i n t id = item.getItemId();

//noinspection SimplifiableIfStatement i f (id ==R.id.action s e t t i n g s ) { r e t u r n true;

76 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

}

r e t u r n super.onOptionsItemSelected(item); } p r i v a t e void connectWebSocket() { URI u r i ; t r y { u r i = new URI(”ws://192.168.1.52:1338/”); } catch (URISyntaxException e) { e.printStackTrace() ; r e t u r n; }

mWebSocketClient = new WebSocketClient(uri) { @Override p u b l i c void onOpen(ServerHandshake serverHandshake) { Log . i (”Websocket”,”Opened”); isConnected = true; JSONObject jsonCommand = new JSONObject() ; mWebSocketClient. send(”Hello from ” + Build. MANUFACTURER +” ” + Build .MODEL) ; t r y { jsonCommand. put(”event”,”phone”); } catch (JSONException e) { e.printStackTrace() ; } mWebSocketClient. send(jsonCommand. toString () ) ; }

@Override p u b l i c void onMessage(String s) {

boolean newTitle = true; String tempTitle; t r y { JSONObject json = new JSONObject(s); Log.d(TAG, json.toString()); s = json.getString(”event”); i f (json .optJSONObject(”data”) != null) { tempTitle = json .getJSONObject(”data”). g e t S t r i n g (”title”); i f(title .equals(tempTitle)) { newTitle = false; } title = json.getJSONObject(”data”). g e t S t r i n g (”title”); posterURL = new URL(json .getJSONObject(” data”).getString(”poster”)); }

} catch (JSONException e1) { Log . e (TAG,”onMessage JSONException”); } catch (MalformedURLException e) { e.printStackTrace() ; }

77 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

i f (newTitle) { System.out. println(”New t i t l e”); new AsyncTask() { @Override p r o t e c t e d Void doInBackground(Void ... params ) { t r y { InputStream in = new URL( posterURL. toString()). openStream() ; posterArt = BitmapFactory. decodeStream(in); } catch (Exception e) { // log error } r e t u r n null; }

@Override p r o t e c t e d void onPostExecute(Void result ) { System.out. println(”onPostExecute”); i f(playing) { p l a y i n g= true; setNotification(title ,”Accedo VIA”); } e l s e { p l a y i n g= false; setNotification(title ,”Accedo VIA”); }

} } .execute(); } e l s e { i f (s.equals(”webPlay”)) { p l a y i n g = true; setNotification(title ,”Accedo VIA”); } e l s e if (s.equals(”webPause”)) { p l a y i n g = false; setNotification(title ,”Accedo VIA”); } } }

@Override p u b l i c void onClose(int i, String s, boolean b) { Log . i (”Websocket”,”Closed ”+ s); isConnected = false; }

@Override p u b l i c void onError(Exception e) { Log . i (”Websocket”,”Error ” + e.getMessage());

78 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

isConnected = false; } } ; mWebSocketClient. connect() ; }

p u b l i c class ResponseReceiver extends BroadcastReceiver { p u b l i c static final S t r i n g ACTION RESP = ”MESSAGE PROCESSED”;

@Override p u b l i c void onReceive(Context context , Intent intent) { String command = intent.getStringExtra( ActionIntentHandler .COMMAND) ; System.out. println(”Connected to websocket: ”+ isConnected);

JSONObject jsonCommand = new JSONObject() ;

i f (command != null && isConnected) { t r y { jsonCommand. put(”event”, command. toString()) ; } catch (JSONException e) { e.printStackTrace() ; }

System.out. println(”jsonCommand: ” + jsonCommand .toString());

mWebSocketClient. send(jsonCommand. toString () ) ; i f (command . e q u a l s (”phonePlay”)) p l a y i n g = true; e l s e if (command . e q u a l s (”phonePause”)) p l a y i n g = false; setNotification(title ,”Accedo VIA”); }

} } }

Listing D.2: ActionIntentHandler.java, ’phone’ module handling the Intents package com.example.gustavnoren.androidmaize; import android.app.IntentService; import android.content.Intent; import android.os.Build; import android.util.Log; import org.java websocket. client .WebSocketClient; import org.java websocket .handshake.ServerHandshake;

79 APPENDIX D. ANDROID WEAR NOTIFICATION EXTENDER

import java.net.URI; import java.net.URISyntaxException;

/∗∗ ∗ Created by gustavnoren on 20/02/15. ∗/ p u b l i c class ActionIntentHandler extends IntentService { p u b l i c static final String TAG =”Debug... ActionHandler”; p u b l i c static final S t r i n g COMMAND =”command”;

p u b l i c ActionIntentHandler() { super(”WebSocketService”); }

@Override p u b l i c void onCreate() { super.onCreate() ; Log . d (TAG,”onCreate”); }

@Override p r o t e c t e d void onHandleIntent(Intent intent) { Intent broadcastIntent = new Intent(); broadcastIntent . setAction(MainActivity.ResponseReceiver. ACTION RESP) ;

i f (intent.getExtras() != null) { i f (intent.getExtras().get(”command”).equals(”play”) ) { Log . d (TAG,”getExtras: play”); broadcastIntent . putExtra(COMMAND,”phonePlay”); } e l s e if(intent.getExtras().get(”command”).equals(” pause”)) { broadcastIntent . putExtra(COMMAND,”phonePause”); } e l s e if(intent.getExtras().get(”command”).equals(” volUp”)) { broadcastIntent . putExtra(COMMAND,”phoneVolUp”); } e l s e if(intent.getExtras().get(”command”).equals(” volDown”)) { broadcastIntent . putExtra(COMMAND,”phoneVolDown” ); } } e l s e { broadcastIntent . putExtra(COMMAND,””); } broadcastIntent . addCategory( Intent .CATEGORY DEFAULT) ; sendBroadcast(broadcastIntent); } }

80 Appendix E

Apple Watch

Listing E.1: InterfaceContoller.h, The header file of the InterfaceController in the WatchKit Extension // InterfaceController.h // Maize WatchKit Extension // // Created by Gustav Beck−Noren on 10/02/15. // Copyright(c) 2015 Gustav Beck −Noren. All rights reserved.

#import #import #import

@interface InterfaceController : WKInterfaceController @property (weak, nonatomic) IBOutlet WKInterfaceButton ∗ playButton ; @property (weak, nonatomic) IBOutlet WKInterfaceButton ∗ volumeUpButton ; @property (weak, nonatomic) IBOutlet WKInterfaceButton ∗ volumeDownButton ; @property (weak, nonatomic) IBOutlet WKInterfaceLabel ∗ nowPlayingLabel ; @property (weak, nonatomic) IBOutlet WKInterfaceLabel ∗ playingInfo ; @property (weak, nonatomic) IBOutlet WKInterfaceGroup ∗mainGroup ; @property (weak, nonatomic) NSString ∗ t i t l e ; @end

Listing E.2: InterfaceContoller.m, The implementation file of the Interface- Controller in the WatchKit Extension // InterfaceController.m // Maize WatchKit Extension //

81 APPENDIX E. APPLE WATCH

// Created by Gustav Beck−Noren on 10/02/15. // Copyright(c) 2015 Gustav Beck −Noren. All rights reserved.

#import”InterfaceController.h”

@interface InterfaceController()

@end

@implementation InterfaceController

SRWebSocket ∗ webSocket ; NSDictionary ∗ jsonDictionary ; NSData ∗ jsonData ; NSCache ∗ posterCache ; NSString ∗ description; bool isPlaying = false;

− (void)awakeWithContext:( id)context { [super awakeWithContext: context ];

// Configure interface objects here. NSLog(@”InterfaceController awake”); }

− (void)willActivate { // This method is called when watch view controller is about to be visible to user [super willActivate];

i f (!isPlaying) [[ self playButton]setBackgroundImage :[UIImage imageNamed :@”playicon.png”]]; e l s e [[ self playButton]setBackgroundImage :[UIImage imageNamed :@”pauseicon.png”]];

[ self connectWebSocket]; }

− (void)didDeactivate { // This method is called when watch view controller is no l o n g e r visible [super didDeactivate]; }

− (void)connectWebSocket { i f(webSocket == nil) { webSocket.delegate = nil; webSocket = nil;

NSString ∗ urlString = @”ws://192.168.1.52:1338”; SRWebSocket ∗newWebSocket = [[SRWebSocket alloc ] initWithURL :[NSURL URLWithString: urlString ]]; newWebSocket.delegate = self ;

82 APPENDIX E. APPLE WATCH

[newWebSocket open ]; } }

#pragma mark − SRWebSocket delegate

− (void )webSocketDidOpen :( SRWebSocket ∗)newWebSocket { webSocket = newWebSocket; [webSocket send :[ NSString stringWithFormat:@”Hello from %@”, [UIDevice currentDevice ].name]];

jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @”phone”,@”event”, n i l ] ; NSError ∗ e r r o r = n i l ; jsonData = [NSJSONSerialization dataWithJSONObject: jsonDictionary options :NSJSONWritingPrettyPrinted error :& e r r o r ] ; NSString ∗ jsonString = [[NSString alloc] initWithData: jsonData encoding :NSUTF8StringEncoding ];

[webSocket send: jsonString ]; }

− (void)webSocket :(SRWebSocket ∗)webSocket didFailWithError :( NSError ∗) e r r o r { [ self connectWebSocket]; }

− (void)webSocket :(SRWebSocket ∗)webSocket didCloseWithCode:( NSInteger)code reason :(NSString ∗)reason wasClean :(BOOL) wasClean { [ self connectWebSocket]; }

− (void)webSocket :(SRWebSocket ∗)webSocket didReceiveMessage :( id ) message { // NSLog(@”Received message:%@”, message);

jsonData = [message dataUsingEncoding:NSUTF8StringEncoding ]; jsonDictionary = [NSJSONSerialization JSONObjectWithData: jsonData options:0 error:nil ];

NSString ∗ blurString = @”http://blurmaster.apps.accedo.tv/ b l u r?r=8&q=”; NSString ∗command = [jsonDictionary objectForKey:@”event”]; NSString ∗posterURL = [NSString stringWithFormat:@”%@%@”, blurString ,[[ jsonDictionary objectForKey:@”data”] objectForKey :@”poster”]];

i f(![ title isEqualToString :([[ jsonDictionary objectForKey:@ ”data”]objectForKey:@”title”])]) { NSURL ∗imageURL = [NSURL URLWithString:posterURL ]; NSData ∗imageData = [NSData dataWithContentsOfURL: imageURL ] ; UIImage ∗ image = [UIImage imageWithData:imageData ];

83 APPENDIX E. APPLE WATCH

[[ self playingInfo] setText: t i t l e ] ;

//[[ self mainGroup] setBackgroundImage:image]; }

self . title = [[jsonDictionary objectForKey:@”data”] objectForKey :@”title”]; description = [[ jsonDictionary objectForKey:@”data”] objectForKey :@”description”];

i f ([command isEqualToString :@”webPlay”]) { i s P l a y i n g = true; [[ self playButton]setBackgroundImage :[UIImage imageNamed :@”pauseicon.png”]]; [[ self nowPlayingLabel] setText: t i t l e ] ; } e l s e if ([command isEqualToString :@”webPause”]) { i s P l a y i n g = false; [[ self playButton]setBackgroundImage :[UIImage imageNamed :@”playicon.png”]]; }

}

−(void)sendPlay { jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @”phonePlay”,@”event”, n i l ] ; NSError ∗ e r r o r = n i l ; jsonData = [NSJSONSerialization dataWithJSONObject: jsonDictionary options :NSJSONWritingPrettyPrinted error :& e r r o r ] ; NSString ∗ jsonString = [[NSString alloc] initWithData: jsonData encoding :NSUTF8StringEncoding ];

[webSocket send: jsonString ]; isPlaying = TRUE; } −(void)sendPause { jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @”phonePause”,@”event”, n i l ] ; NSError ∗ e r r o r = n i l ; jsonData = [NSJSONSerialization dataWithJSONObject: jsonDictionary options :NSJSONWritingPrettyPrinted error :& e r r o r ] ; NSString ∗ jsonString = [[NSString alloc] initWithData: jsonData encoding :NSUTF8StringEncoding ];

[webSocket send: jsonString ]; isPlaying = FALSE; } −(void )sendVolumeUp { jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @”phoneVolUp”,@”event”, n i l ] ; NSError ∗ e r r o r = n i l ;

84 APPENDIX E. APPLE WATCH

jsonData = [NSJSONSerialization dataWithJSONObject: jsonDictionary options :NSJSONWritingPrettyPrinted error :& e r r o r ] ; NSString ∗ jsonString = [[NSString alloc] initWithData: jsonData encoding :NSUTF8StringEncoding ];

[webSocket send: jsonString ]; } −(void )sendVolumeDown { jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @”phoneVolDown”,@”event”, n i l ] ; NSError ∗ e r r o r = n i l ; jsonData = [NSJSONSerialization dataWithJSONObject: jsonDictionary options :NSJSONWritingPrettyPrinted error :& e r r o r ] ; NSString ∗ jsonString = [[NSString alloc] initWithData: jsonData encoding :NSUTF8StringEncoding ];

[webSocket send: jsonString ]; }

− (IBAction)playAction { i f(isPlaying) { [[ self playButton]setBackgroundImage :[UIImage imageNamed :@”playicon.png”]]; [ self sendPause]; i s P l a y i n g = false; } e l s e { [[ self playButton]setBackgroundImage :[UIImage imageNamed :@”pauseicon.png”]]; [self sendPlay]; i s P l a y i n g = true; } }

− (IBAction)volumeUpAction { [ self sendVolumeUp]; } − (IBAction)volumeDownAction { [ self sendVolumeDown]; }

−(void)updateButton {

} @end

85 APPENDIX E. APPLE WATCH

86 P˚asvenska Detta dokument h˚allstillg¨angligtp˚aInternet – eller dess framtida ers¨attare – under en l¨angretid fr˚anpubliceringsdatum under f¨oruts¨attningatt inga extra-ordin¨araomst¨andigheteruppst˚ar. Tillg˚angtill dokumentet inneb¨artillst˚andf¨orvar och en att l¨asa,ladda ner, skriva ut enstaka kopior f¨orenskilt bruk och att anv¨anda det of¨or¨andrat f¨orickekommersiell forskning och f¨orundervisning. Overf¨oringav¨ upphovs- r¨attenvid en senare tidpunkt kan inte upph¨ava detta tillst˚and. All annan anv¨andningav dokumentet kr¨aver upphovsmannens medgivande. F¨oratt garantera ¨aktheten,s¨akerheten och tillg¨angligheten finns det l¨osningarav teknisk och administrativ art. Upphovsmannens ideella r¨attinnefattar r¨attatt bli n¨amndsom upphovs- man i den omfattning som god sed kr¨aver vid anv¨andningav dokumentet p˚a ovan beskrivna s¨attsamt skydd mot att dokumentet ¨andraseller presenteras i s˚adanform eller i s˚adant sammanhang som ¨arkr¨ankande f¨orupphovsman- nens litter¨araeller konstn¨arligaanseende eller egenart. F¨orytterligare information om Link¨opingUniversity Electronic Press se f¨orlagetshemsida http://www.ep.liu.se/

In English The publishers will keep this document online on the Internet – or its possible replacement – for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copy- right owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be pro- tected against infringement. For additional information about the Link¨opingUniversity Electronic Press and its procedures for publication and for assurance of document in- tegrity, please refer to its WWW home page: http://www.ep.liu.se/

c Gustav Beck-Nor´en