Using Remotely Executing Software Via a Mobile Device
Total Page：16
File Type：pdf, Size：1020Kb
Using Remotely Executing Software via a Mobile Device Vipul Delwadia Stuart Marshall Ian Welch School of Mathematics, Statistics and Computer Science Victoria University of Wellington, PO Box 600, Wellington 6140, Email: (vipul | stuart | ian)@mcs.vuw.ac.nz Abstract bile devices. Our subsequent analysis and experi- mentation with our prototype and related works will There are scenarios in mobile computing that may be framed by the particular requirement that the benefit from separating presentation from computa- user should get “instantaneous” response to their tion. Traditionally this separation can be achieved actions. We define “instantaneous response” to be via tools such as VNC. However such factors as net- within 100ms (Nielsen 1993). The preliminary exper- work latency and additional communication overhead iments we present at the end of this paper attempt to can slow down the presentation of a remotely execut- measure whether our prototype and other existing re- ing mobile application below acceptable performance lated tools can support this response time for various levels, especially for domains like gaming where re- values of network connectivity and distance. sponses may need to appear to be instantaneous. We present RemoteMe, an architecture and Java-based prototype for mobile-client / server communication 2 Motivation that only requires a very thin mobile client. We hy- pothesise that RemoteMe will support faster response There are a number of scenarios where we see such a times to user input than existing software solutions method proving useful. such as VNC. This paper presents a preliminary anal- Firstly, while mobile device technology has ad- ysis of our first prototype, and experimentally com- vanced significantly in the past few years, they still pares it to an open-source mobile-based VNC system. have a relatively low computational power. This means that there are some tasks that may be beyond the ability of the mobile device to perform by itself, Keywords: Mobile Devices, Remote Gaming, Human even though the device has sufficient screen real es- Computer Interaction tate and input mechanisms to handle the necessary input and output. One possible solution to this is to divide interaction and computation between the mo- 1 Introduction bile ”client” and a more powerful server. An example of this is a user playing a computer game that requires Mobile devices are now extremely commonplace in significant and computationally expensive AI. developed countries. Increasingly, mobile devices are Secondly, mobile devices by their very nature open being used for tasks other than human-to-human up opportunities in providing location-specific ser- communication, and support tasks previously asso- vices (Hinze & Buchanan 2006). In this scenario, ciated with more powerful desktop computers or users could walk into a particular area (such as a servers. We propose a method for provisioning mobile museum) and access applications that the museum devices with applications that are accessed through supplies to provide more information or interactive the mobile device while in fact executing remotely on features. The user would not need to already have a server connected over wifi. or install a client specific to the museum, and could While this is — on the surface — similar to exist- interact with the applications as if they were native ing client-server architectures, our method would not applications on their mobile. require an application-specific client on the mobile de- Thirdly, there are situations where the owner of vice. This lack of an application-specific client is remi- an application wishes to make the application avail- niscent of how web-based applications can be accessed able to users, but does not want to distribute (and solely through a standard web browser. However, our potentially lose control of) the software directly. An method would also provide a greater ability for in- example of this could be a user walking into a game dividual mobile devices to tailor the user interface store and evaluating the latest mobile game to hit to those devices’ specific input/output mechanisms. the market by playing the full version on their de- This is useful because different mobile devices have vice, without the store losing control of the product. different screen sizes, different keypads, and different button mappings / locations for controls in a given mobile application user interface. 3 Related Work In this paper we present our prototype and archi- tecture for a system that will support a very-thin- In the original vision of thin client computing, such client approach to remote software access via mo- as the Java applet-based architecture (Gosling et al. 1996), the clients need only provide an execution plat- Copyright c 2009, Australian Computer Society, Inc. This pa- form and applications can be downloaded on demand per appeared at the Tenth Australasian User Interface Con- and run locally. In fact, many mobile phone providers ference (AUIC2009), Wellington, New Zealand. Conferences use exactly this technique to distribute mobile phone in Research and Practice in Information Technology (CRPIT), Vol. 93. Paul Calder, Gerald Weber, Eds. Reproduction for games and other applications. The downside of this academic, not-for profit purposes permitted provided this text technique is the complexity of the game that can be is included. played is bounded by the capabilities of the mobile phone that is executing it. Therefore this related performance enhancements over slow networks, and work focuses on only providing the presentation layer more recent versions of X11 system have been opti- on the phone. In particular, remote desktop software mized such that LBX no longer gave any real benefits and video streaming software. (Packard & Gettys 2003). 3.1 Remote Desktop Software 3.2 Video Remote desktop software allow a user of a less pow- Aside from the main remote desktop protocols dis- erful machine to access their desktop that runs on cussed above, researchers are exploring the use of a more powerful machine across a network. In our streaming video to remotely control a system. An context, this would allow a user to run a game on example application of this are the model helicopters a remote machine but use it as if it was running on trialled by the Merseyside Police in Britain. An op- their mobile device. There is a large number of remote erator controls the helicopter and receives feedback desktop software applications available for a range of on the helicopter’s surroundings via CCTV cameras platforms including Windows, Linux, Mac OS X, em- mounted on the helicopter itself (BBC 2007). bedded devices and mobile devices. However, most (Zhuang & Wang 2006) have developed an “IP- use variations of one of the following protocols for based real time video monitoring system with con- communication: Remote Desktop Protocol (RDP), trollable platform”. Their approach was to have a Remote Frame Buffer (RFB) and X11. camera and microphone connected to an embedded system that compresses the incoming data and sends RDP RDP (MSDN 2008) is the technology un- it to a server. When a client comes along, they request derlying Microsoft’s remote desktop software for the the stream from the server. Upon receiving the com- Windows Server and Client operating systems. Once pressed data, the client decompresses and displays it. a connection is made between an RDP client and Cell phones, along with standard desktop machines, server, the server sends commands to render windows are supported as clients. to the client, which is a mixture of drawing commands and images. User input via keyboard/mouse is sent 4 RemoteMe back to the server from the client accordingly. In this section, we present the RemoteMe prototype RFB RFB is the protocol that VNC (Virtual Net- and the underlying architecture. work Computing) is built on top of. The RFB pro- tocol operates in a client/server manner as well, but 4.1 Design the server sends the display to the client as rectangu- lar snapshots of the remote desktop (or frame buffer). The RemoteMe architecture/prototype describes a The snapshots are simply bitmaps representing rect- set of connected components on both the client and angular sections of the remote desktop. Instead of the server. These components facilitate monitoring sending just the raw bitmaps, the rectangles can be an application executing on the server-side, sending encoded in a number of ways. Each specific encoding the required instructions to the mobile device for ren- requires both client and server support, and one such dering, and receiving user input from the mobile de- encoding, supported by TightVNC (TightVNC 2008), vice for processing by the server-side application. is the Tight encoding with optional JPEG compres- Figure 1 shows our architecture, with specific Java- sion. Essentially, Tight performs compression on the based technologies included. The architecture as- rectangles before they are sent, reducing the amount sumes a wifi network connection between the client of traffic generated. and the server. The server runs a mobile phone emulator system (such as the Sun Wireless Emulator) in which a mo- X11 X11 is a client-server system, where the server bile application is executed. Prior to execution, ad- is the application rendering the window, and the ditional code (in the form of AspectJ aspects (Kicza- client is the application requesting to be rendered. les et al. 2001)) is woven into the mobile application When a window is to be rendered, the X11 client ap- source code. The additional code is responsible for plication sends the X11 server a request to create a monitoring the use of standard UI libraries within window with an id. Further, when the client wishes the targeted mobile application and is the same for to fill that window with content, it refers to it by id. all monitored applications, although the weaving it- There have been at least two attempts at trying self would be application-dependent and automated.