[retrospective]

Mahadev Satyanarayanan Carnegie Mellon University

Editor: Carla Schlatter Ellis A Brief History of Cloud Offload A Personal Journey from Odyssey Through Cyber Foraging to Cloudlets

very time you use a voice command also relevant to the concept of cloudlets [18] is identified as a key long-term constraint of on your smartphone, you are that has emerged as an important theme mobile computing. To quote verbatim from benefitting from a technique called in mobile-cloud convergence. Reflecting that piece entitled “Mobile Computing” [15]: Ecloud offload.Your speech is captured my participation in this evolution from its by a microphone, pre-processed, then sent origins, this article is a personal account of Mobile elements are resource-poor over a wireless network to a cloud service the key developments in this research area. relative to static elements. Regardless that converts speech to text. The result is It focuses on mobile computing, of future technological advances, a then forwarded to another cloud service or ignoring many other uses of remote mobile unit’s weight, power, size, and sent back to your mobile device, depending execution since the 1980s such as ergonomics will always render it less on the application. Speech recognition distributed processing, query processing, computationally capable than its static and many other resource-intensive mobile distributed object systems, and distributed counterpart. While mobile elements services require cloud offload. Otherwise, partitioning. will undoubtedly improve in absolute the service would be too slow and drain too ability, they will always be at a relative much of your battery. At the Dawn of Mobile Computing disadvantage. Research projects on cloud offload In 1993, the editor of the “Hot Topics” are hot today, with MAUI [4] in 2010, department of IEEE Computer invited me Later, in 1995, I made this same point Odessa [13] and CloneCloud [2] in 2011, to write a short thought piece on the brand in a keynote talk entitled “Fundamental and COMET [8] in 2012. These build on new topic of mobile computing that was Challenges of Mobile Computing,” to the a rich heritage of work dating back to just starting to emerge. To the best of my ACM Principles of Distributed Systems the mid-1990s on a theme that is broadly knowledge this is the first place where the Conference. An invited paper based on this characterized as cyber foraging. They are inherent resource poverty of mobile devices talk was published in the following year’s

OCTOBER 2014 | Volume 18, Issue 4 GetMobile 19 [RETROSPECTIVE]

conference proceedings [16] and has been FIGURE 1: Evolution of Hardware Performance widely cited since. In the 20+ years since that prediction Typical Server Typical Handheld was made, it has remained consistently or Wearable true. Figure 1 from Flinn [5] illustrates Year Processor Speed Device Speed the consistent large gap in the processing 1997 Pentium® II 266 MHz Palm Pilot 16 MHz power of typical server and mobile device 2002 Itanium® 1 GHz Blackberry 133 MHz hardware over a 16-year period. This 5810 stubborn gap reflects a fundamental reality of user preferences. The most sought- after 2007 Intel® 9.6 GHz Apple 412 MHz CoreTM 2 (4 cores) iPhone features of a mobile device are light weight, small size, and long battery life. By using 2011 Intel® 32 GHz Samsung 2.4 GHz remote execution on static infrastructure Xeon® X5 (2x6 cores) Galaxy S2 (2 cores) that does not suffer from these constraints, 2013 Intel® 64 GHz Samsung 6.4 GHz the mobile device can overcome its Xeon® E5 (2x12 cores) Galaxy S4 (4 cores) computational limitations. Google Glass 2.4 GHz OMAP 4430 (2 cores) A Decade of Exploration We began exploring remote execution for (Source: adapted from Flinn [5]) mobile computing in the context of the Odyssey system. As described in 1997 by Noble et al [12], the Janus speech recognition application was modified to operate in one of Vision and Challenges” [17] suggested envisioned in this article encompassed both three modes in Odyssey. One mode involved many research directions that have remote execution and the staging of data strictly local execution. The second mode since proved fruitful. Abstracting and nearby. The data staging concept was later involved strictly remote execution: the speech generalizing from the work on remote expanded by Flinn et al [7]. Today’s content signal captured on the mobile client was execution, it emphasized the enduring delivery networks (CDNs) can be viewed shipped to a remote server for recognition, problem of resource poverty of mobile as implementing a form of data staging. and the transcribed text was shipped back. devices and suggested that leveraging The work on fluid replication by Noble The third mode was hybrid: a preliminary nearby resources in a principled way might et al [11, 3] is related to the concept of phase of speech processing was done locally, be the best approach to addressing this cyber foraging for data, and the associated and the extracted information was shipped problem. I used the metaphor of foraging infrastructure concept of WayStations can to a remote server for the completion of the for food in the wild: be viewed as similar to surrogates. recognition process. An important attribute The period from 2001-2008 saw vigorous of this implementation was Odyssey’s ability Cyber foraging, construed as “living off research activity in this space. A detailed to dynamically select the optimal execution the land”, may be an effective way to account of these efforts is provided in the mode based on runtime factors such as deal with this problem. The idea is to excellent survey by Flinn [5]. current network bandwidth. Odyssey was dynamically augment the computing thus the technical forerunner of today’s resources of a wireless mobile The Cloud Appears mobile speech-to-text systems such as Siri, as computer by exploiting wired hardware A nagging question from the very well as the recent mechanisms for adaptive infrastructure. beginning that had never been satisfactorily cloud offload that were mentioned at the … answered was who would provide the beginning of the paper. When hardware in the wired infrastructure for cyber foraging? How Rudenko et al [14] were the first to infrastructure plays this role, we call it would trustworthy hardware for remote suggest that remote execution could a surrogate of the mobile computer it is execution be dispersed in the environment? extend battery life on mobile devices, and temporarily assisting. How would mobile devices discover them? to provide experimental evidence for this What business incentives would there be to claim. At about the same time, Flinn [6] Implicit in the foraging metaphor is the deploy and maintain such infrastructure? extended Odyssey to support energy- notion of “nearby” resources. It seemed The emergence of aware adaptation and showed that remote obvious that low latency and high bandwidth circa 2008 suddenly clarified these issues. execution could indeed save energy in the to the remote execution site was essential. Independent of mobile computing Janus speech recognition application. However, the paper did not explicitly discuss considerations, companies like Amazon In 2001, I was invited to write an article proximity as an important criteria. This were making computing resources that for an IEEE Personal Communications turned out to have unexpected consequences could be used for transient purposes special issue on pervasive computing. This some years later, as described below. It is available on the Internet. There was now article, entitled “Pervasive Computing: important to note that cyber foraging as a business model and hence incentive for

20 GetMobile OCTOBER 2014 | Volume 18, Issue 4 [RETROSPECTIVE]

deploying and maintaining hardware for remote execution. The emergence of Apple’s cloud-based Siri speech recognition service both validated the remote execution concept for mobile devices at commercial scale, and simultaneously offered a viable deployment model. “In the cloud,” was the obvious answer to the question, “Where should remote execution be performed?” In industry and in academia, the imminent convergence of mobile computing and cloud computing was heralded. Alas, that celebration was premature. The economics of cloud computing strongly favor the centralization of infrastructure into a few large data centers. It is through economies of scale in operations and system administration that cloud computing wins. Unfortunately, global consolidation implies large average separation between a mobile device and its cloud. End-to-end communication then involves many network hops and results in FIGURE 2. Two-Level Cloud-Cloudlet Architecture high latencies. From the beginning, I had implicitly assumed low latency and high bandwidth between mobile device and remote execution site. But by early 2008, I realized that my implicit assumption of the need for a two-level architecture for similar to replacing a networking element “nearby” in framing the cyber foraging mobile-cloud convergence. such as router. We described the cloudlet concept was a mistake. I should have made Figure 2 illustrates this architecture. concept in a paper that has proved to explicit the importance of proximity. The first level is today’s unmodified cloud highly influential, receiving more than 550 Discussions with a number of senior infrastructure. The second level consists citations in less than five years [18]. Cisco’s researchers in mobile computing at the of dispersed elements with no hard state recent concept of fog computing [1] is 2008 MobiSys conference in Breckenridge, called cloudlets. A cloudlet is effectively a consistent with the cloudlet concept. CO convinced me that it was necessary “second-class ” with soft state to make the case for proximity explicit to generated locally or cached from the first Proximity Really Matters the research community and to industry. level. By using persistent caching instead Alas, I discovered that just publishing a In close collaboration, from of hard state, the management of cloudlets paper articulating the need for proximity , Roy Want from Intel, Ramon is kept simple in spite of their physical in remote execution was not sufficient. Caceres from AT&T, Nigel Davies from dispersal at the edge of the Internet. There were still plenty of skeptics out there Lancaster University, and I articulated Replacing a cloudlet is conceptually who needed much harder evidence that proximity was really necessary. Perhaps the best evidence of this skepticism is this verbatim quote from the 2009 panel summary of my proposal on cloudlets THE EMERGENCE OF APPLE’S CLOUD-BASED that was rejected by the National Science SIRI SPEECH RECOGNITION SERVICE BOTH Foundation: VALIDATED THE REMOTE EXECUTION Many panelists do not agree with the premise of the proposal in which CONCEPT FOR MOBILE DEVICES AT distant cloud computing incurs too high latency to be acceptable by mobile COMMERCIAL SCALE, AND SIMULTANEOUSLY applications. They question the validity of such assumption as the proposal OFFERED A VIABLE DEPLOYMENT MODEL. provides no real data to justify it. . . .

OCTOBER 2014 | Volume 18, Issue 4 GetMobile 21 [RETROSPECTIVE]

milliseconds (a)

code is run on the mobile device. In spite of avoiding the energy and performance cost of Wi-Fi communication, the data shows that mobile-only does worse than using the cloudlet. Offloading is clearly important for these applications.

Closing Thoughts milliseconds Although simple in concept, cloudlets are a disruptive force in mobile computing. (b) Face Recognition Their ability to provide low latency, high bandwidth access to energy-unlimited FIGURE 3. Response Time Distribution and Per-Operation Energy Cost high-end computing within one wireless (Source: Ha et al [10]) hop of mobile devices is transformative. Many valuable applications can be created using cloudlets. What new kinds of mobile services might To gain the necessary hard evidence, transmitted back to the mobile device. cloudlets make possible in the future? One we invested substantial effort in obtaining For face recognition, the identity of the possibility is wearable cognitive assistance, latency-sensitive and compute-intensive person is returned. Each curve in Figure as described by Ha et al [9]. Such a service applications such as face recognition, 3 corresponds to the CDF of the observed combines the video capture and sensing object recognition, and augmented reality. response time distribution. The ideal curve capabilities of wearable devices such as Extensive measurements have now proven is a step function that jumps to 1.0 at the Google Glass with cloud offload to perform beyond all reasonable doubt that running origin. Figure 3 shows that this ideal is best real-time scene analysis. To a user in such applications on a nearby cloudlet approximated by a cloudlet. cognitive decline (for example, an elderly gives much better response time and lower End-to-end latency plays a dominant person with Alzheimer’s disease), such a energy usage on a mobile device than role, as shown by the worsening response service could offer helpful assistance, much running them in the cloud. time curves corresponding to more distant as a GPS navigation system does today for a The importance of cloudlets can be AWS locations. Increasing response time driver in an unfamiliar city. Crisp response seen in the results shown in Figure 3 for also increases the per-operation energy time, below a few tens of milliseconds augmented reality and face recognition consumption on the mobile device. This per offload operation, will be essential for on a mobile device. Full details of these value is shown beside the corresponding avoiding user distraction. Cloudlets will be experiments and many others can be found label in the middle of the figure. For essential for such a service. in the paper by Ha et al [10]. An image from example, the mobile device consumes 1.1 J The early stages of the convergence of the mobile device (located in Pittsburgh, on average to perform an augmented reality mobile computing and cloud computing PA) is transmitted over a Wi-Fi first hop to operation on the cloudlet, but 3.1 J, 5.1 J, are already under way. There is a small a cloudlet or to an Amazon Web Services and so on when performing it on AWS- window of opportunity to shape this (AWS) data center. East, AWS-West, etc. Although these results convergence so that it preserves openness The image is processed at the were obtained on AWS, similar results can and cohesiveness of software interfaces and destination by computer vision code be expected with any offload service that is network protocols in the new infrastructure executing within a virtual machine (VM). concentrated in a few large data centers. that will emerge. This path can lead to For augmented reality, buildings The label “mobile-only” inFigure 3 the kind of explosive growth seen in the in the image are recognized and labels corresponds to a case where no offloading Internet itself. The next few years will be corresponding to their identities are is performed, and the computer vision critical in shaping this future. Q

22 GetMobile OCTOBER 2014 | Volume 18, Issue 4 [RETROSPECTIVE]

Cognitive VMs Wireless Face recognition connection Control VM Object Recognition l Context Inference (MOPED) Wearable device ontro sorc Sen Object Recognition (STF) Device PubSub Video/Acc/GPS/… Comm OCR sensor streams UPnP Motion classifier Us er assi Sensor flows stan Activity Inference ce Cognitive flows User Guidance VM Augmented Reality VM boundary … many other Cloudlet cognitive VMs here…

FIGURE 4. Cloudlet architecture of Gabriel, a wearable cognitive assistance system (Source: Ha et al [9])

REFERENCES [1] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli. Satyanarayanan. Data Staging on Untrusted In Proceedings of the 16th ACM Symposium on Fog Computing and Its Role in the Internet of Surrogates. In Proceedings of FAST’03: 2nd Operating Systems Principles, Saint-Malo, France, Things. In Proceedings of the First Edition of the USENIX Conference on File and Storage October 1997. MCC Workshop on Mobile Cloud Computing, Technologies, San Francisco, CA, March 2003. [13] M. Ra, A. Sheth, L. Mummert, P. Pillai, D. Helsinki, Finland, 2012. [8] M. S. Gordon, D. A. Jamshidi, S. Mahlke, Z. Wetherall, and R. Govindan. Odessa: Enabling [2] B.-G. Chun, S. Ihm, P. Maniatis, M. Naik, and M. Mao, and X. Chen. COMET: Code Ooad Interactive Perception Applications on Mobile A. Patti. CloneCloud: Elastic Execution between by Migrating Execution Transparently. In 10th Devices. In Proceedings of the International Mobile Device and Cloud. In Proceedings of USENIX Symposium on Operating Systems Design Conference on Mobile Systems, Applications, and EuroSys 2011, Salzburg, Switzerland, April 2011. and Implementation (OSDI 12), Hollywood, CA, Services (MobiSys 2011), Bethesda, MD, June [3] L. P. Cox and B. D. Noble. Fast Reconciliations October 2012. 2011. in Fluid Replication. In Proceedings of the The [9] K. Ha, Z. Chen, W. Hu, W. Richter, P. Pillai, and [14] A. Rudenko, P. Reiher, G. J. Popek, and G. H. 21st International Conference on Distributed M. Satyanarayanan. Towards Wearable Cognitive Kuenning. Saving portable computer battery Computing Systems, 2001. Assistance. In Proceedings of the Twelfth power through remote process execution. Mobile [4] E. Cuervo, A. Balasubramanian, D.-k. Cho, A. International Conference on Mobile Systems, Computing and Communications Review, 2(1), Wolman, S. Saroiu, R. Chandra, and P. Bahl. Applications, and Services, Bretton Woods, NH, January 1998. MAUI: Making Smartphones Last Longer with June 2014. [15] M. Satyanarayanan. Mobile computing. IEEE Code Ooad. In Proceedings of the 8th International [10] K. Ha, P. Pillai, G. Lewis, S. Simanta, S. Clinch, Computer, 26(9), September 1993. Conference on Mobile Systems, Applications, and N. Davies, and M. Satyanarayanan. The Impact [16] M. Satyanarayanan. Fundamental Challenges Services, San Francisco, CA, June 2010. of Mobile Multimedia Applications on Data in Mobile Computing. In Proceedings of the [5] J. Flinn. Cyber Foraging: Bridging Mobile and Center Consolidation. In Proceedings of the IEEE ACM Symposium on Principles of Distributed Cloud Computing via Opportunistic Offload. International Conference on Cloud Engineering, Computing, Ottawa, Canada, 1996. Morgan & Claypool Publishers, 2012. San Francisco, CA, March 2013. [17] M. Satyanarayanan. Pervasive Computing: [6] J. Flinn and M. Satyanarayanan. Energy- [11] B. Noble, B. Fleis, M. Kim, and J. Zajkowski. Vision and Challenges. IEEE Personal aware Adaptation for Mobile Applications. In Fluid replication. In Proceedings of the Network Communications, 8(4), 2001. Proceedings of the 17th ACM Symposium on Storage Symposium, Seattle, WA, October 1999. [18] M. Satyanarayanan, P. Bahl, R. Caceres, and Operating Systems and Principles, Kiawah Island, [12] B. Noble, M. Satyanarayanan, D. Narayanan, N. Davies. The Case for VM-Based Cloudlets in SC, December 1999. J. Tilton, J. Flinn, and K. Walker. Agile Mobile Computing. IEEE Pervasive Computing, [7] J. Flinn, S. Sinnamohideen, N. Tolia, and M. Application-Aware Adaptation for Mobility. 8(4), October-December 2009.

OCTOBER 2014 | Volume 18, Issue 4 GetMobile 23