DOCSLIB.ORG

Thesis Proposes a New Type of Mobile/Cloud Operating System Designed to Meet the Evolving Needs of Modern Applications

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Thesis Proposes a New Type of Mobile/Cloud Operating System Designed to Meet the Evolving Needs of Modern Applications ©Copyright 2017 Irene Y. Zhang Towards a Flexible, High-Performance Operating System for Mobile/Cloud Applications Irene Y. Zhang A dissertation submitted in partial fulllment of the requirements for the degree of Doctor of Philosophy University of Washington 2017 Reading Committee: Henry M. Levy, Chair Arvind Krishnamurthy Luis Ceze Program Authorized to Oer Degree: Computer Science & Engineering University of Washington Abstract Towards a Flexible, High-Performance Operating System for Mobile/Cloud Applications Irene Y. Zhang Chair of the Supervisory Committee: Director Henry M. Levy Computer Science & Engineering ¿e convergence of ubiquitous mobile devices, large-scale cloud platforms and pervasive network con- nectivity have changed the face of modern user applications. Unlike a traditional desktop application, which runs on a single machine and supports a single user, the typical user-facing application today spans numerous mobile devices and cloud servers while supporting large numbers of users. ¿is shi signicantly increased the diculty of building new user applications. Programmers must now confront challenges introduced by distributed deployment (e.g., partial failures), new mobile/cloud application features (e.g., reactivity), and new mobile/cloud requirements (e.g., scalability). ¿is thesis proposes a new type of mobile/cloud operating system designed to meet the evolving needs of modern applications. Mobile/cloud applications are the standard applications of the future; thus, they deserve a rst-class operating system that simplies their development and run-time management. Our key contribution is the design, implementation and evaluation of three systems that together form the basis for a new mobile/cloud operating system: (1) Sapphire, a new distributed run-time and process management system, (2) Diamond, a new distributed memory management system, and (3) TAPIR, a new distributed storage system. Each system introduces new operating systems abstractions and mechanisms designed to eliminate the challenges and simplify the develop- ment of mobile/cloud applications. We demonstrate that, like operating systems of the past, these systems make it easier for programmers to build bigger and more complex applications. Table of Contents Page List of Figures...............................................v Glossary.................................................. ix Chapter 1: Overview........................................1 1.1 ¿e Mobile/Cloud Revolution................................2 1.2 ¿e Case for a Mobile/Cloud Operating System.....................3 1.3 Mobile/Cloud Challenges and Application Requirements................6 1.3.1 Mobile/Cloud OS Requirements..........................6 1.3.2 ¿e Challenge of Designing a Mobile/Cloud OS................8 1.4 Existing Mobile/Cloud Systems...............................9 1.4.1 Distributed Storage Systems............................ 11 1.4.2 Service-Oriented Architectures.......................... 13 1.4.3 Distributed Run-time Systems........................... 14 1.4.4 Backend-as-a-Service................................ 16 1.5 Contributions......................................... 18 1.5.1 Run-time Management: Sapphire......................... 18 1.5.2 Memory Management: Diamond......................... 20 1.5.3 Storage Management: TAPIR........................... 21 1.5.4 Summary....................................... 22 Chapter 2: Sapphire......................................... 24 2.1 Background.......................................... 26 i 2.2 Architecture.......................................... 27 2.2.1 Design Goals..................................... 27 2.2.2 System Architecture................................. 28 2.3 Programming Model..................................... 29 2.4 Deployment Kernel...................................... 33 2.5 Deployment Managers.................................... 34 2.5.1 DM Library...................................... 35 2.5.2 DM Structure and API............................... 36 2.5.3 DM Code Example................................. 39 2.5.4 DM Design Examples................................ 42 2.6 Implementation........................................ 45 2.7 Experience & Evaluation................................... 47 2.7.1 Applications..................................... 47 2.7.2 Experimental Setup................................. 52 2.7.3 Microbenchmarks.................................. 52 2.7.4 Deployment Manager Performance........................ 54 2.8 Related Work.......................................... 57 2.9 Summary............................................ 58 Chapter 3: Diamond........................................ 60 3.1 Background.......................................... 62 3.1.1 Roll-your-own Data Management......................... 62 3.1.2 Wide-area Storage Systems............................. 64 3.1.3 Reactive Programming Frameworks....................... 65 3.2 Architecture & Programming Model............................ 65 3.2.1 System Model.................................... 65 3.2.2 Data Model...................................... 66 3.2.3 System Calls..................................... 68 3.2.4 Reactive Data Management Guarantees..................... 70 3.2.5 A Simple Code Example.............................. 71 3.2.6 Oine Support.................................... 73 3.2.7 Security........................................ 74 3.3 System Design......................................... 74 ii 3.3.1 Data Management Architecture.......................... 75 3.3.2 rmap and Language Bindings............................ 75 3.3.3 Transaction Coordination Overview....................... 75 3.4 Wide-area Optimizations.................................. 79 3.4.1 Data-type Optimistic Concurrency Control................... 79 3.4.2 Client Caching with Bounded Validity Intervals................ 80 3.4.3 Data Push Notications............................... 82 3.5 Experience & Evaluation................................... 82 3.5.1 Prototype Implementation............................. 82 3.5.2 Programming Experience............................. 83 3.5.3 Performance Evaluation.............................. 86 3.6 Related Work.......................................... 91 3.7 Summary............................................ 93 Chapter 4: TAPIR.......................................... 95 4.1 Background.......................................... 97 4.2 Inconsistent Replication................................... 99 4.2.1 IR Overview..................................... 99 4.2.2 IR Protocol...................................... 103 4.2.3 Correctness...................................... 111 4.3 Building Atop IR....................................... 114 4.3.1 IR Application Protocol Requirement: Invariant checks must be performed pairwise. ........................................ 114 4.3.2 IR Application Protocol Requirement: Application protocols must be able to change consensus operation results. ...................... 115 4.3.3 IR Performance Principle: Application protocols should not expect operations to execute in the same order. ............................ 116 4.3.4 IR Performance Principle: Application protocols should use cheaper incon- sistent operations whenever possible rather than consensus operations. ... 116 4.4 TAPIR.............................................. 117 4.4.1 Overview....................................... 117 4.4.2 Protocol........................................ 119 4.4.3 Correctness...................................... 130 4.5 TAPIR Extensions....................................... 135 iii 4.5.1 Read-only Transactions............................... 135 4.5.2 Serializability..................................... 137 4.5.3 Synchronous Log Writes.............................. 137 4.5.4 Retry Timestamp Selection............................. 138 4.5.5 Tolerating Very High Skew............................. 138 4.6 Evaluation........................................... 138 4.6.1 Experimental Setup................................. 139 4.6.2 Single Datacenter Experiments.......................... 141 4.6.3 Wide-Area Latency................................. 142 4.6.4 Abort and Retry Rates................................ 144 4.6.5 Comparison with Weakly Consistent Systems.................. 145 4.7 Related Work.......................................... 145 4.7.1 Replication...................................... 146 4.7.2 Distributed Transactions.............................. 147 4.8 Summary............................................ 148 Chapter 5: Conclusion....................................... 149 5.1 Looking Forward: ¿e Path to Adoption.......................... 150 5.2 Concluding Remarks..................................... 151 Bibliography................................................ 152 Appendix A: Open-source Code................................... 169 Appendix B: TLA+ Specication.................................. 170 B.1 Inconsistent Replication Specication........................... 170 B.2 TAPIR Specication...................................... 194 iv List of Figures Figure Number Page 1.1 ¿e evolution of application environments and system architectures. ..........4 1.2 Evaluation of existing systems. We graph each class of systems based on how many mobile/cloud requirements it meets. If the system gives applications control over the requirement or requires applications help meet the requirement, then we draw the requirements as a range between the number of requirements
Load more

Recommended publications
  • Improved Density-Based Spatio–Textual Clustering on Social Media
    SUBMITTED TO IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 1 Improved Density-Based Spatio–Textual Clustering on Social Media Minh D. Nguyen and Won-Yong Shin, Senior Member, IEEE Abstract—DBSCAN may not be sufficient when the input data type is heterogeneous in terms of textual description. When we aim to discover clusters of geo-tagged records relevant to a particular point-of-interest (POI) on social media, examining only one type of input data (e.g., the tweets relevant to a POI) may draw an incomplete picture of clusters due to noisy regions. To overcome this problem, we introduce DBSTexC, a newly defined density-based clustering algorithm using spatio–textual information. We first characterize POI-relevant and POI-irrelevant tweets as the texts that include and do not include a POI name or its semantically coherent variations, respectively. By leveraging the proportion of POI-relevant and POI-irrelevant tweets, the proposed algorithm demonstrates much higher clustering performance than the DBSCAN case in terms of F1 score and its variants. While DBSTexC performs exactly as DBSCAN with the textually homogeneous inputs, it far outperforms DBSCAN with the textually heterogeneous inputs. Furthermore, to further improve the clustering quality by fully capturing the geographic distribution of tweets, we present fuzzy DBSTexC (F-DBSTexC), an extension of DBSTexC, which incorporates the notion of fuzzy clustering into the DBSTexC. We then demonstrate the robustness of F-DBSTexC via intensive experiments. The computational complexity of our algorithms is also analytically and numerically shown. Index Terms—Density-based clustering, fuzzy clustering, geo-tagged tweet, point-of-interest (POI), spatio–textual information.
    [Show full text]
  • The Twitter API
    Social Networking and Micro-Blogging Application Development: The Twitter API Nisaar Jagroep Roland Brouwer Oxalis Atindriyaratri LIACS LIACS LIACS [email protected] [email protected] [email protected] Keywords: Social Media, Micro- blogging, Twitter, question which might escape attention is : What is a status Communication update? The answer to this question is dependent on the user interpretation. However, most often experiences, events, announcement, are considered to be one (e.g. ABSTRACT “Having a discussion with Mark about Hegel”, “I can’t In this report we will explore the social blogging service: come home, stupid ashcloud….” or “Today is my Twitter. We shall describe concisely what Twitter is, birthday!”). In an ideal Twitter case, if one has experienced what it is able to, and give a technical overview with a something, or wants to express an opinion that is ‘how to start’ guide. Overall, we shall look critical influencing this person daily activities, one should post against strengths and weaknesses of this web technology. these messages on Twitter, called “Tweets”. Theoretically every experience in daily life that can be considered as a status update, should be ‘tweeted’ by the user. Also it is 1. PURPOSE, CONTEXT AND HISTORY possible to define status update partially by asking: “What are you doing?” and “What is happening or what is going to [16] Twitter, also called “the SMS of the internet” , is a social happen?”. Still it should be noted that tweets contains far micro-blogging service. This service allows users to put text more, then answers on these questions, and are in essence on the internet with a maximum of 140 characters, similar not limited by any rule.
    [Show full text]
  • What Is Black Twitter? a Rhetorical Criticism of Race, Dis/Information, and Social Media
    CHAPTER 5. WHAT IS BLACK TWITTER? A RHETORICAL CRITICISM OF RACE, DIS/INFORMATION, AND SOCIAL MEDIA Alexandria Lockett Spelman College Key Words and Concepts: Black Rhetoric, Blackness, Surveillance, Disinformation, Information Warfare, Hashtags “I TWITTER, THEREFORE I AM BLACK, OR AM I BLACK AND THEREFORE I TWITTER?” Black Twitter tells a powerful story about emerging digital racial literacies and their relationship to a long history of anti-Black information warfare. Through a constellation of hashtags, Black Twitter maps onto archived demonstrations of Black cultural discourse. Its users critically exchange stories about bearing witness to the absurdity of race and racism. These include having humorous dis- cussions about shared racial experiences (#Growingupblack and #Thanksgiving- withBlackFamilies), critiquing Black mis/representation in media and entertain- ment (#OscarsSoWhite and #CNNBeLike], putting racist White folks on blast (#AskRachel and #PaulasBestDishes], and making structural anti-Black racism more publicly visible (#Icantbreathe, #NotJustUVA, #reclaimOSU, and #Black- LivesMatter]. Such hashtags have generated countless conversation threads that appeal to a vast global network of (presumably and predominantly) Black social media users.1 Throughout this chapter, I argue that Black Twitter should be read as a text comprised of narratives about digitally mediated Black cultural preservation and expression. I will show how Black Twitter serves as a living archive of collective 1 #BlackLivesMatter has been the most extensively studied hashtag that is associated with Black Twitter. According to one report, “[#BlackLivesMatter] has been used nearly 30 million times on Twitter – an average of 17,002 times per day – as of May 1, 2018.” (Anderson et al. Activism in the Social Media Age).
    [Show full text]
  • Twarfing: Malicious Tweets Morton Swimmer Costin G
    Click to edit Master title style • Click to edit Master text styles – Second level • Third level – Fourth level » Fifth level Twarfing: Malicious tweets Morton Swimmer Costin G. Raiu Trend Micro Kaspersky Lab June 10th, 2009 Virus Bulletin 2009 – SeptemberEvent details 24th (title,, Geneva place) Thanks to: • Special thanks (Costin): – Selma Ardelean: GUI+statistics – Dan Demeter: daemon , downloader , scanning – Alexandru Tudorica: DB design, URL fetching, expansion, scanning – Stefan Tanase – suggestions and web 2.0 expertise (you can watch his presentation tomorrow morning in the Corp stream) •Spp()ecial thanks (Morton) – Rainer Link (architecture) – David Sancho (URL expansion) June 10th, 2009 Event details (title, place) Overview • What is Twitter? • Malware on Twitter – Notable incidents • The link: Twitter and URL shortening services • Twitter and the Google SB API • RbtRobots: – Kaspersky Architecture and Statistics – Trend Architecture and Statistics • Conclusions June 10th, 2009 Event details (title, place) What is Twitter? • Publish/Subscribe Communications system • Founded by Jack Dorsey, Biz Stone and Evan Willia m s back in 2 006 • SMS/Website, WebService (API) • Subscribers can read from this Browser • Push App Phone • SSSMS: Phone • Pull • Web site: Browser • WS API: Application App Browser Phone • RSS: Application June 10th, 2009 Event details (title, place) Related to: • Instant Messaging/XMPP •Is manyyy to many, but best with small g gproups or one-to-one • Twitter similar, but publish/subscriber model more persistent
    [Show full text]
  • Performance Analysis of Opportunistic Content Distribution Via Data-Driven Mobility Modeling
    Performance Analysis of Opportunistic Content Distribution via Data-Driven Mobility Modeling LJUBICA PAJEVI∆ Doctoral Thesis Stockholm, Sweden, 2018 School of Electrical Engineering TRITA-EECS-AVL-2018:76 and Computer Science ISBN 978-91-7729-980-6 KTH, Stockholm, Sweden Akademisk avhandling som med tillstånd av Kungl Tekniska högskolan framlägges till offentlig granskning för avläggande av doktorexamen onsdagen den 21 november 2018 i Hörsal F3, Kungl Tekniska högskolan, Lindstedtsvägen 26, Stockholm. © Ljubica PajeviÊ, November 2018 Tryck: Universitetsservice US AB Abstract For the most part of the developed world, ubiquitous connectivity has become a norm in human daily lives. Globally, the access to information on the Internet and telecommunication services has been recognized as one of the accelerators of human progress, promoting the right to education and facilitating access to globally shared knowledge. The increasing capacity and user experience demands on the one side, and the need for increased net- work coverage and the availability of affordable telecommunication services on the other side, pose significant challenges for the current network archi- tectures. While the next generation wireless technologies and the pervasive deployment of the currently available communication systems are poised to solve the above challenges, the main inhibitors of immediate action are still socio-economical factors. Due to the pervasiveness of end-user mobile devices, device-to-device communication (D2D) can be leveraged as an intermediate— and a complementary—solution. D2D communication enables networking in an opportunistic manner. An opportunistic network is formed by co-located mobile users in order to ex- change data via direct wireless links when their devices are within transmis- sion range, without relying on the use of fixed network infrastructure.
    [Show full text]
  • How to Successfully Market an API Fine-Tuning Developer Relations & Platform Advocacy
    How to Successfully Market an API Fine-Tuning Developer Relations & Platform Advocacy Nordic APIs This book is for sale at http://leanpub.com/how-to-market-an-API This version was published on 2016-09-29 This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do. © 2016 Nordic APIs Contents Preface ............................. i Part One: Planning ................ 1 Building from the Ground Up: Tips for Starting Your API Program .......................... 2 Clarify Your Needs .................... 3 Get Buy In (From Everyone) ............... 4 Aim for a Public MVP ................... 5 Act on Feedback ...................... 7 Build your Practice .................... 8 Final Thoughts ....................... 9 Define Your Target Developer Audience .......... 11 Why Create a Developer “Persona”? . 12 The Developer Brain ................... 13 But Plenty of Other People Are Interested in APIs, Too! 14 Expanding our Portal: Developer End User Evangelism . 15 Varying Industry Backgrounds . 16 Location & Demographics . 18 API Use Cases ....................... 18 Technology Preferences . 19 Lessen The Corporate Branding . 20 Developer Experience ................... 21 Build it And They Will _____ . 22 CONTENTS Understand Your Audience . 22 Developer Experience is a Key Ingredient of Quality APIs 23 API Model Canvas – offspring of Lean Canvas . 24 Developers are the Rockstars of the API Economy . 25 Addressing the Entire API Model Canvas . 27 Case Study: National Library of Finland . 27 MVP for next step ..................... 28 Gain speed and make it fun . 29 Final Thoughts ....................... 29 Part Two: Developer Relations ....
    [Show full text]
  • BJMC IV SEM Course-XVII Social Media Social Networking Sites A
    BJMC IV SEM Course-XVII Social Media Social Networking Sites A social networking service is a platform to build social networks or social relations among people who share interests, activities, backgrounds or real-life connections. A social network service consists of a representation of each user (often a profile), his or her social links, and a variety of additional services. Social networks are web-based services that allow individuals to create a public profile, to create a list of users with whom to share connections, and view and cross the connections within the system. Most social network services are web-based and provide means for users to interact over the Internet, such as e-mail and instant messaging. Social network sites are varied and they incorporate new information and communication tools such as mobile connectivity, photo/video/sharing and blogging. Online community services are sometimes considered as a social network service, though in a broader sense, social network service usually means an individual-centered service whereas online community services are group-centered. Social networking sites allow users to share ideas, pictures, posts, activities, events, interests with people in their network. The main types of social networking services are those that contain category places (such as former school year or classmates), means to connect with friends (usually with self-description pages), and a recommendation system linked to trust. Popular methods now combine many of these, with American-based services such as Facebook,
    [Show full text]
  • Opinions Libres - 1 / 14 - Edition PDF Du 13 Septembre 2020 2
    1 Retour en Silicon Valley (3/7) grands de l’Internet Pour cette troisième partie de ce compte-rendu détaillé d’une visite d’une semaine dans la Silicon Valley, nous allons couvrir quatre grands acteurs rencontrés sur place :Facebook , Twitter, Google et Zynga. Sachant que plus ces entreprises sont grandes, moins on apprend de choses sur elles en les visitant ! A cela plusieurs raisons : on les connait assez bien car on exploite leurs outils tous les jours, elles font régulièrement la une des médias, elles ont une offre très large (Google) et le temps de la visite chez elles est très minuté. Il faut aussi dire que plus grande est l’entreprise, plus elle a industrialisé l’organisation de ces visites du fait d’une demande très soutenue. La semaine précédente à notre visite, Google, Facebook et Twitter avaient reçu ainsi la visite d’Eric Besson, Ministre de l’Industrie et de l’Economie Numérique ! Autre donnée de contexte : ces entreprises demandaient la signature d’un NDA (contrat de confidentialité), sauf Twitter. Les photos étaient interdites dans les locaux sauf chez Facebook. Toutes sauf Zynga nous ont reçus avec des interlocuteurs français. Et toutes ces entreprises étaient motivées pour recruter des étudiants. Les photos de ces visites, plutôt extérieures, sont disponibles sur mon site photo. Facebook Barack Obama avait visité Facebook la veille de votre passage, intervenant dans un “Town Hall Meeting” pour les employés du réseau social et quelques invités triés sur le volet. Mais son intervention était générique et ne portait pas spécifiquement sur le numérique. Au même moment, nous étions à quelques pas dans l’incubateur BlackBox situé comme Facebook dans Palo Alto, au point que nous avons entendu puis vu passer les hélicoptères dont un devait bien être le Marine One du président ! Pour ce qui nous concernait chez Facebook, c’était plus simple.
    [Show full text]
  • Szekeres Washington 0250E 2
    DESIGNING STORAGE AND PRIVACY-PRESERVING SYSTEMSFOR LARGE-SCALE CLOUD APPLICATIONS by Adriana SZEKERES A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Washington 2020 Reading committe: Henry M. Levy, Co-Chair Arvind Krishnamurthy, Co-Chair Dan R. K. Ports Program Authorized to Offer Degree: Computer Science & Engineering ©Copyright 2020 Adriana SZEKERES University of Washington Abstract Designing Storage and Privacy-Preserving Systems for Large-Scale Cloud Applications Adriana Szekeres Chair of Supervisory Committee: Henry M. Levy, Arvind Krishnamurthy Computer Science & Engineering Today’s mobile devices sense, collect, and store huge amounts of personal information, which users share with family and friends through a wide range of cloud applications. Many of these applications are large-scale – they must support millions of users from all over the world. These applications must manage a massive amount of user-generated content, they must ensure that this content is always available, and they must protect users’ privacy. To deal with the complexity of managing a large amount of data and ensure its availability, large-scale cloud applications rely on transactional, fault-tolerant, distributed storage systems. Latest hardware advancements and new design trends, such as in-memory storage, kernel-bypass message processing, and geo-distribution, have significantly improved the transaction processing times. However, they have also exposed overheads in the protocols used to implement these systems, which hinder their performance; specifically, existing protocols require unnecessary cross-datacenter and cross-processor coordination. This thesis introduces Meerkat, a new replicated storage system that avoids all cross-core and cross-replica coordination for transactions that do not conflict.
    [Show full text]