Cross Platform Communication Apps
Total Page:16
File Type:pdf, Size:1020Kb
Copyright © IEEE, 2016. This is the author's copy of a paper that appears in an IEEE conference proceeding. Please cite as follows: K. Singh and J. Buford, "Developing WebRTC-based team apps with a cross-platform mobile framework", In proceedings of IEEE Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA, Jan 2016. Developing WebRTC-based Team Apps with a Cross-Platform Mobile Framework Kundan Singh John Buford Avaya Labs Research Avaya Labs Research Santa Clara, CA, USA Basking Ridge, NJ, USA [email protected] [email protected] Abstract—We present lessons learned in developing cross described in other studies for: real-time multi-party streaming platform multi-party team applications. Our apps include a of video using WebRTC, real-time notifications, e.g., using range of communication and collaboration scenarios: document WebSocket, secure access to cloud services, and apps and content sharing in a team space, an agent-based meeting interoperability with browser-based and desktop versions. We helper, phone number dialer via a voice-over-IP (VoIP) gateway, address challenges including and beyond these requirements. and multi-party call in peer-to-peer or client-server mode. We use web real-time communication (WebRTC) to enable the audio These apps include: (1) a team collaboration app for and video media paths in the apps. We use frameworks such as persistent sharing of content with escalation to real-time voice, Chrome Apps and Apache Cordova to create apps that can be video or app sharing, (2) a web-based video call/conference accessed from a browser, or installed on a desktop, mobile service that runs all the app logic in the endpoint, (3) a meeting device, or wearable. The challenges and techniques described in helper agent which joins the conference bridge during a our paper related to audio, video, network, power conservation meeting, automatically creates the meeting summary at the and security are important to other developers building cross- end, and sends it to the participants, (4) three softphones using platform apps involving WebRTC, VoIP and cloud services. Voice-over IP (VoIP) services for small or large businesses, and (5) a serverless video phone using local network multicast. Keywords—HTML5, Apache Cordova, Chrome Apps, WebRTC, Mobile, Cloud, Wearable Our work answers these questions important to those who are developing with or evaluating tools for cross-platform I. INTRODUCTION mobile app development: (1) how to best realize write-once- Team apps have features that require multi-way media run-anywhere for apps that leverage features common to team streaming, multipoint notifications, resource sharing, use of apps, and (2) what are important takeaways from our range of cloud services, security, and operation on a range of devices. apps’ development experience to help existing and new app Underlying technologies used to implement these features such developers. We list several challenges and techniques related to as WebRTC and client-side certificates have inconsistent audio, video, network, power conservation and security. support, particularly across mobile platforms. The capabilities Remainder of this paper is organized as follows. Section II are also important for mobile app developers in categories contains related work on hybrid apps and WebRTC. Section III including gaming, social networking, healthcare and education. describes the apps under consideration. Sections IV and V Due to the plethora of devices, app developers prefer cross- describe the cross platform development process including for platform development with a write-once run-anywhere model. wearable. Sections VI to VIII have our lessons learned on Prior studies have shown that cross-platform development tools HTML5, WebRTC and mobile apps. Section IX describes for mobile offer the economies of write-once run anywhere but power conservation and security of notifications. Section X has are challenging in terms of creating a native user experience our conclusions and future work. (UX) or using device specific capabilities and advanced browser-specific features. However, prior work does not cover II. RELATED WORK AND BACKGROUND the technologies important to team collaboration apps. The leading mobile application stores, Google Play and iOS App Store, each have more than 1M mobile apps [7]. We have developed a set of cross-platform applications Powerful cross-platform mobile app development frameworks using the Apache Cordova (PhoneGap) [1] hybrid app [8] have emerged so that developers can produce hybrid native development framework. These applications are implemented applications from HTML5 – PhoneGap (Apache Cordova) and in HTML5 and typically use cloud services from our self- Sencha Touch – or cross-platform tools and SDKs, such as service cloud portal [2][3], including WebRTC (web real-time Adobe AIR, MoSync, Appcelerator, and Corona. communication) [4] and WebSocket [5][6]. Malavolta et al. [9] study user perception of cross-platform In this paper we discuss challenges and lessons for cross- mobile apps by mining 3M reviews of 11,917 free apps from platform development relevant to classes of apps that involve the Google Play Store. Of the 11K apps, about 500 were real-time communication and cloud-based services. The apps developed with cross-platform mobile frameworks, and more that we describe have important requirements beyond those than 50% of these with Apache Cordova [1]. Their results show that cross-platform development is most frequently used for data intensive apps and least used for apps with platform Camera Mi Web <video/> interaction, e.g., games and audio. They conclude that hybrid getUserMedia Server Play development frameworks are perceived as better suited for Publish #1 #1 RTCPeer RTCPeer data-intensive mobile apps than for dealing with low-level, Connection Connection platform-specific features, and in some categories, native apps Play #2 Peer-to-peer media path #2 Publish getUserMedia are viewed better due to performance and less bugs. <audio/> STUN/ Heitkötter et al. [10] found that Appcelerator based mobile TURN Microphone apps appear more native, whereas PhoneGap apps are like web Fig. 1. WebRTC conceptual elements and client server system sites. Angulo and Ferre [11] evaluated the user experience using the SIP proxy and gateway. Separating the app user’s comparing natively developed app with an equivalent app built authentication from the EDP token simplifies billing to that using a cross-platform tool. A native iOS UX is more difficult customer, instead of its individual app users. to obtain using cross-platform tools compared to Android apps. 2) IP office phone: Avaya IP office is a communication system Several studies have shown the feasibility of cross-platform for small and midsize businesses, which we host on cloud for mobile app development, e.g., RSS reader using Appcelerator customer trial. The IP office phone is a client to connect to this [12], geolocation and map APIs using PhoneGap [13][14], or VoIP system to make or receive voice or video calls. It uses WebSocket with PhoneGap [15]. Cross-platform apps using client-server media path anchored at the server (Fig.2b). This is device features such as accelerometers and multi-touch gesture not uncommon in cloud telephony, e.g., for media services of could be comparable to native versions of the same apps [16]. recording, interactive voice response or telephony gateway. Unlike previous studies, we focus on real-time 3) Vclick: This is a pure web-based multi-party client to enable communication and collaboration, and the strength of our work video call, conferencing, video presence, text chat, shared is in the recommendations about developer practices, the white-board, notepad and screen sharing [17]. It does not experience with WebRTC, and the number of apps. depend on legacy VoIP systems or browser plugins. It works as a collection of loosely coupled apps that mash up at the data WebRTC [4] refers to the ongoing efforts by W3C, IETF level instead of pair-wise app permissions, e.g., white-board and browser vendors to enable web pages to exchange real- app can be joined from a separate device in an existing desktop time media streams. A page can capture from local mic and/or video call (Fig.2c). The cross-platform version is created as a camera using getUserMedia as a local media stream abstraction, single app, with reduced functions of voice, video and text create RTCPeerConnection, a peer-to-peer abstraction between chat, but no other sharing apps. Unlike traditional server-side browser instances, and send a media stream from one browser web apps, Vclick runs all the app logic in the client, and uses a to another (Fig.1). RTCPeerConnection emits certain signaling “thin” resource server [6] that enables data storage and pub/sub data such as session description and transport addresses, which events, e.g., a client subscribed to a resource gets notified when must be sent and applied to the RTCPeerConnection at the other it changes. A persistent client-server connection enables shared browser to establish a media path. Web pages typically use data access and incoming events such as call invite. It creates WebSocket [5] or Ajax over HTTPS between the browser and peer-to-peer (full-mesh) media paths during a call. the web server to do such session negotiations. Browsers can use the web page supplied STUN or TURN servers [4] to 4) Connected Spaces: This is a team collaboration system for discover server reflexive and/or relayed addresses for the persistent sharing of content, e.g., documents, meeting notes, media paths that cross network boundaries. wiki or calendar [18]. Long lived distributed groups can organize their work in spaces, e.g., a space for “Third Floor III. MULTI-PARTY TEAM APPS Construction” project contains all the relevant content of that Team apps involve both active or synchronous modes such project. Users can create persistent annotations on shared as video or teleconference, as well as passive or asynchronous content (Fig.2d) or start impromptu audio, video and/or text modes such as SharePoint or a wiki.