DOCSLIB.ORG

C 2012 Brian Cho

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
C 2012 Brian Cho c 2012 Brian Cho SATISFYING STRONG APPLICATION REQUIREMENTS IN DATA-INTENSIVE CLOUD COMPUTING ENVIRONMENTS BY BRIAN CHO DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Urbana-Champaign, 2012 Urbana, Illinois Doctoral Committee: Associate Professor Indranil Gupta, Chair Professor Tarek F. Abdelzaher Assistant Professor P. Brighten Godfrey Dr. Marcos K. Aguilera, Microsoft Research Silicon Valley Abstract In today's data-intensive cloud systems, there is a tension between resource limitations and strict requirements. In an effort to scale up in the cloud, many systems today have unfortunately forced users to relax their requirements. However, users still have to deal with constraints, such as strict time dead- lines or limited dollar budgets. Several applications critically rely on strongly consistent access to data hosted in clouds. Jobs that are time-critical must re- ceive priority when they are submitted to shared cloud computing resources. This thesis presents systems that satisfy strong application requirements, such as consistency, dollar budgets, and real-time deadlines, for data-intensive cloud computing environments, in spite of resource limitations, such as band- width, congestion, and resource costs, while optimizing system metrics, such as throughput and latency. Our systems cover a wide range of environments, each with their own strict requirements. Pandora gives cloud users with deadline or budget constraints the optimal solution for transferring bulk data within these constraints. Vivace provides applications with a strongly consis- tent storage service that performs well when replicated across geo-distributed data centers. Natjam ensures that time-critical Hadoop jobs immediately re- ceive cluster resources even when less important jobs are already running. For each of these systems, we designed new algorithms and techniques aimed at making the most of the limited resources available. We implemented the systems and evaluated their performance under deployment using real-world data and execution traces. ii To dad, mom, and David. Thank you for your constant love and support. iii Acknowledgments I would like to thank my advisor, Indranil Gupta. Thank you for your guidance and belief in me. Our meetings have always lifted up my spirits and productivity. I will miss them. I am grateful to my committee members, Marcos Aguilera, Brighten God- frey, and Tarek Abdelzaher. The preliminary and final exams were especially rewarding because of their sharp insights. I would like to thank the members of DPRG. Ramses Morales, Jay Patel, and Steve Ko inspired me, each in their distinct ways. Watching them, I shaped the habits that carried me throughout my research. Discussions in the office with Imranul Hoque, Muntasir Rahman, and Simon Krueger provided new insights and encouragement that pushed my projects along. I am thankful to have collaborated with many brilliant, fun folks. The aforementioned members of DPRG were a constant presence, and I had the privilege to work on projects big and small with all of them. Marcos's theoretical insights and persistent day to day hard work were inspiring. I can only hope some of it rubbed off on me during our work together. I picked up a lot of system building while working with Abhishek Verma and Nicolas Zea. Abhishek has remained a reliable sounding board through- out my research. I had fun working with Liangliang Cao, Hyun Duk Kim, Min-Hsuan Tsai, Zhen Li, ChengXiang Zhai, and Thomas Huang. Espe- cially, Liangliang and Hyun Duk showed great persistence in presenting our work to the world. I'd like to thank Cristina Abad and Nathan Roberts for their expertise and hard work that led to successful large-scale cluster exper- iments. Special thanks are due to Derek Dagit for his timely support with the CCT cluster. I would like to thank my internship mentors: Mike Groble at Motorola, Sandeep Uttamchandani at IBM Almaden, Nathan at Yahoo, and Marcos at iv Microsoft Research Silicon Valley. Finally, I'd like to thank family and friends. Creating a list of personal acknowledgments would be an impossible task, even more so than creating the list of academic acknowledgments listed here (where I have undoubtedly missed many important people). I omit such a list. I hope instead that God blesses me with the opportunity to extend my gratitude in person to each special individual. v Table of Contents Chapter 1 Introduction . 1 1.1 Thesis and Contributions . 2 1.2 Related Work . 7 Chapter 2 Pandora-A: Deadline-constrained Bulk Data Transfer via Internet and Shipping Networks . 9 2.1 Motivation . 9 2.2 Problem Formulation . 13 2.3 Deadline-constrained Solution . 17 2.4 Optimizations . 22 2.5 Experimental Results . 30 2.6 Related Work . 43 2.7 Summary . 44 Chapter 3 Pandora-B: Budget-constrained Bulk Data Transfer via Internet and Shipping Networks . 45 3.1 Motivation . 45 3.2 Problem Formulation . 48 3.3 Budget-constrained Solution . 48 3.4 Experimental Results . 57 3.5 Related Work . 65 3.6 Summary . 66 Chapter 4 Vivace: Consistent Data for Congested Geo-distributed Systems . 67 4.1 Motivation . 67 4.2 Setting and Goals . 69 4.3 Design and Algorithms . 71 4.4 Analysis . 82 4.5 Implementation . 85 4.6 Evaluation . 85 4.7 Related Work . 94 4.8 Summary . 96 vi Chapter 5 Natjam: Strict Queue Priority in Hadoop . 97 5.1 Motivation . 97 5.2 Solution Approach and Challenges . 100 5.3 Design: Overview . 101 5.4 Design: Background . 102 5.5 Design: Suspend/resume Architecture . 105 5.6 Design: Implementation . 111 5.7 Design: Eviction Policies . 115 5.8 Design: Extensions . 117 5.9 Evaluation . 120 5.10 Related Work . 133 5.11 Summary . 135 Chapter 6 Conclusions and Future Directions . 137 References . 138 vii Chapter 1 Introduction Cloud computing has taken off as a multi-billion dollar market, worth $40.7 billion in 2010, and predicted to grow up to $241 billion by 2020 [116]. This growth has been fueled by the emergence of cloud infrastructures where cus- tomers pay on-the-go for limited resources such as CPU hours used, net- work data transfer, and disk storage [6, 17, 26, 65], rather than paying an upfront cost [44]. The workloads in cloud infrastructures are largely data- intensive. For example, web applications such as photo sharing or social net- work services store, interpret, and propagate large amounts of user-generated data. Other data includes click data, and increasingly scientific or business data. These are routinely processed by parallel data processing program- ming frameworks such as MapReduce/Hadoop [62, 19], Pig Latin [111], and Hive [127], at scales of terabytes and petabytes. In handling this data, there is a fundamental tension between resource limi- tations and user requirements. At the same time, there is a desire to optimize certain runtime metrics. Thus, in this dissertation we design solutions that meet strong user requirements, such as time, money, consistency, and re- source assignment. At the same time, we design our solutions to be practical in resource-limited cloud environments, by optimizing on key metrics. Many of today's cloud offerings were designed to only optimize key metrics within the specific resource limits of their environment, without meeting many strong user requirements. These systems force users to relax critical requirements. For example, the Dynamo key-value store [64] was designed for low latency and high availability, in a failure-prone environment. The users of Dynamo are forced to have data follow only a (weak) eventual consistency model. Yet, there is a pervasive demand from users to have their applications meet a strong requirement. One such requirement is a time deadline. During the 2008 elections, a newspaper reporter turned to cloud computing to quickly 1 Strong user Key optimized System Key resource limitations requirement metric Pandora-A (Ch 2) Deadline Low $ cost Bandwidth, latency, and cost Pandora-B (Ch 3) $ Budget Short transfer time of various transfer options Vivace (Ch 4) Consistency Low latency Bandwidth of cross-site links Natjam (Ch 5) Queue priority High throughput Cluster capacity Figure 1.1: Contributions of the thesis. process the records of a candidate that had just been released [34]. Cloud infrastructure that allowed the reporter to specify and meet deadlines would have been invaluable in publishing the results within the news cycle. Another requirement is a dollar budget. A group of academic researchers who wish to process a large dataset in the cloud should avoid financial loss by using a system that treats their budget (e.g., funds from a grant) as a strong requirement. A third requirement is strong consistency guarantees. A storage system that meets this requirement can make developers of a large-scale messaging application more effective [109]. Finally, a cluster operator can offer shared cluster resources to many users at a time, without worrying about interfer- ence, when the cluster ensures that important jobs are given priority. 1.1 Thesis and Contributions Our work shows that it is feasible to satisfy strong application requirements, such as consistency, dollar budgets, and real-time deadlines, for data-intensive cloud computing environments, in spite of resource limitations, such as band- width, congestion, and resource costs, while simultaneously optimizing run- time metrics, such as throughput and latency. Our contributions, as summarized in Figure 1.1, are the design and imple- mentation of systems that solve problems with strong requirements. These systems are practical, i.e., they feature good runtime performance and opti- mally use limited resources. They cover a wide range of application require- ments, practical features, and resource limitations. Pandora-A and Pandora-B create solutions for transferring bulk data across many shipping and Internet links that have unique bandwidth, latency and dollar costs.
Load more

Recommended publications
  • RAMA: a Filesystem for Massively Parallel Computers
    RAMA: A FILESYSTEM FOR MASSIVELY the advantages of the system, and some possible draw- PARALLEL COMPUTERS backs. We conclude with future directions for research. Ethan L. Miller and Randy H. Katz PREVIOUS WORK University of California, Berkeley Berkeley, California There have been few file systems truly designed for parallel machines. While there have been many mas- sively parallel processors, most of them have used uni- ABSTRACT processor-based file systems. These computers generally perform file I/O through special I/O interfac- es and employ a front-end CPU to manage the file sys- This paper describes a file system design for massively tem. This method has the major advantage that it uses parallel computers which makes very efficient use of a well-understood uniprocessor file systems; little addi- few disks per processor. This overcomes the traditional tional effort is needed to support a parallel processor. I/O bottleneck of massively parallel machines by stor- The disadvantage, however, is that bandwidth to the ing the data on disks within the high-speed interconnec- parallel processor is generally low, as there is only a tion network. In addition, the file system, called single CPU managing the file system. Bandwidth is RAMA, requires little inter-node synchronization, re- limited by this CPU’s ability to handle requests and by moving another common bottleneck in parallel proces- the single channel into the parallel processor. Nonethe- sor file systems. Support for a large tertiary storage less, systems such as the CM-2 use this method. system can easily be integrated into the file system; in fact, RAMA runs most efficiently when tertiary storage is used.
    [Show full text]
  • Curriculum Vitae Ethan L
    Curriculum Vitae Ethan L. Miller June 2021 Computer Science & Engineering Department MOBILE: +1 (831) 295-8432 University of California, Santa Cruz 1156 High Street, MS SOE3 EMAIL: [email protected] Santa Cruz, CA 95064 https://www.soe.ucsc.edu/~elm/ EMPLOYMENT HISTORY 2018–present Professor, Computer Science and Engineering Department, University of California, Santa Cruz 2017–2018 Professor, Computer Engineering Department, University of California, Santa Cruz 2014–2019 Veritas Presidential Chair in Storage [formerly Symantec Presidential Chair in Storage & Security], Jack Baskin School of Engineering, University of California, Santa Cruz 2013–2019 Director, NSF IUCRC Center for Research in Storage Systems, University of California, Santa Cruz 2012–2013 Pure Storage (on 80% leave from the University of California, Santa Cruz) 2009–2013 Site Director, NSF IUCRC Center for Research in Intelligent Storage, University of California, Santa Cruz 2008–2017 Professor, Computer Science Department, University of California, Santa Cruz 2007–present Associate Director, Storage Systems Research Center, University of California, Santa Cruz 2002–2008 Associate Professor, Computer Science Department, University of California, Santa Cruz 2000–2002 Assistant Professor, Computer Science Department, University of California, Santa Cruz 1999 System Architect, Endeca, Cambridge, MA 1994–2000 Assistant Professor, Computer Science and Electrical Engineering Department, University of Mary- land Baltimore County 1988–1994 Research Assistant, Computer Science Division, University of California at Berkeley 1988–1990 Teaching Associate, Computer Science Division, University of California at Berkeley 1987–1988 Software Engineer, BBN Laboratories, Cambridge, MA 1986 Summer intern, GTE Government Systems, Rockville, MD EDUCATION 1995 Ph. D., University of California at Berkeley, Computer Science (advisor: Randy Katz) Thesis: Storage Hierarchy Management for Scientific Computing 1990 M.
    [Show full text]
  • Improving Mapreduce Performance in Heterogeneous Environments
    Improving MapReduce Performance in Heterogeneous Environments Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, Ion Stoica University of California, Berkeley {matei,andyk,adj,randy,stoica}@cs.berkeley.edu Abstract The MapReduce model popularized by Google is very MapReduce is emerging as an important programming attractive for ad-hoc parallel processing of arbitrary data. model for large-scale data-parallel applications such as MapReduce breaks a computation into small tasks that web indexing, data mining, and scientific simulation. run in parallel on multiple machines, and scales easily to Hadoop is an open-source implementation of MapRe- very large clusters of inexpensive commodity comput- duce enjoying wide adoption and is often used for short ers. Its popular open-source implementation, Hadoop jobs where low response time is critical. Hadoop’s per- [2], was developed primarily by Yahoo, where it runs formance is closely tied to its task scheduler, which im- jobs that produce hundreds of terabytes of data on at least plicitly assumes that cluster nodes are homogeneous and 10,000 cores [4]. Hadoop is also used at Facebook, Ama- tasks make progress linearly, and uses these assumptions zon, and Last.fm [5]. In addition, researchers at Cornell, to decide when to speculatively re-execute tasks that ap- Carnegie Mellon, University of Maryland and PARC are pear to be stragglers. In practice, the homogeneity as- starting to use Hadoop for seismic simulation, natural sumptions do not always hold. An especially compelling language processing, and mining web data [5, 6]. setting where this occurs is a virtualized data center, such A key benefit of MapReduce is that it automatically as Amazon’s Elastic Compute Cloud (EC2).
    [Show full text]
  • Ieee-Level Awards
    IEEE-LEVEL AWARDS The IEEE currently bestows a Medal of Honor, fifteen Medals, thirty-three Technical Field Awards, two IEEE Service Awards, two Corporate Recognitions, two Prize Paper Awards, Honorary Memberships, one Scholarship, one Fellowship, and a Staff Award. The awards and their past recipients are listed below. Citations are available via the “Award Recipients with Citations” links within the information below. Nomination information for each award can be found by visiting the IEEE Awards Web page www.ieee.org/awards or by clicking on the award names below. Links are also available via the Recipient/Citation documents. MEDAL OF HONOR Ernst A. Guillemin 1961 Edward V. Appleton 1962 Award Recipients with Citations (PDF, 26 KB) John H. Hammond, Jr. 1963 George C. Southworth 1963 The IEEE Medal of Honor is the highest IEEE Harold A. Wheeler 1964 award. The Medal was established in 1917 and Claude E. Shannon 1966 Charles H. Townes 1967 is awarded for an exceptional contribution or an Gordon K. Teal 1968 extraordinary career in the IEEE fields of Edward L. Ginzton 1969 interest. The IEEE Medal of Honor is the highest Dennis Gabor 1970 IEEE award. The candidate need not be a John Bardeen 1971 Jay W. Forrester 1972 member of the IEEE. The IEEE Medal of Honor Rudolf Kompfner 1973 is sponsored by the IEEE Foundation. Rudolf E. Kalman 1974 John R. Pierce 1975 E. H. Armstrong 1917 H. Earle Vaughan 1977 E. F. W. Alexanderson 1919 Robert N. Noyce 1978 Guglielmo Marconi 1920 Richard Bellman 1979 R. A. Fessenden 1921 William Shockley 1980 Lee deforest 1922 Sidney Darlington 1981 John Stone-Stone 1923 John Wilder Tukey 1982 M.
    [Show full text]
  • Federated Computing Research Conference, FCRC’96, Which Is David Wise, Steering Being Held May 20 - 28, 1996 at the Philadelphia Downtown Marriott
    CRA Workshop on Academic Careers Federated for Women in Computing Science 23rd Annual ACM/IEEE International Symposium on Computing Computer Architecture FCRC ‘96 ACM International Conference on Research Supercomputing ACM SIGMETRICS International Conference Conference on Measurement and Modeling of Computer Systems 28th Annual ACM Symposium on Theory of Computing 11th Annual IEEE Conference on Computational Complexity 15th Annual ACM Symposium on Principles of Distributed Computing 12th Annual ACM Symposium on Computational Geometry First ACM Workshop on Applied Computational Geometry ACM/UMIACS Workshop on Parallel Algorithms ACM SIGPLAN ‘96 Conference on Programming Language Design and Implementation ACM Workshop of Functional Languages in Introductory Computing Philadelphia Skyline SIGPLAN International Conference on Functional Programming 10th ACM Workshop on Parallel and Distributed Simulation Invited Speakers Wm. A. Wulf ACM SIGMETRICS Symposium on Burton Smith Parallel and Distributed Tools Cynthia Dwork 4th Annual ACM/IEEE Workshop on Robin Milner I/O in Parallel and Distributed Systems Randy Katz SIAM Symposium on Networks and Information Management Sponsors ACM CRA IEEE NSF May 20-28, 1996 SIAM Philadelphia, PA FCRC WELCOME Organizing Committee Mary Jane Irwin, Chair Penn State University Steve Mahaney, Vice Chair Rutgers University Alan Berenbaum, Treasurer AT&T Bell Laboratories Frank Friedman, Exhibits Temple University Sampath Kannan, Student Coordinator Univ. of Pennsylvania Welcome to the second Federated Computing Research Conference, FCRC’96, which is David Wise, Steering being held May 20 - 28, 1996 at the Philadelphia downtown Marriott. This second Indiana University FCRC follows the same model of the highly successful first conference, FCRC’93, in Janice Cuny, Careers which nine constituent conferences participated.
    [Show full text]
  • Download and Use Untrusted Code Without Fear
    2 KELEY COMPUTER SCIENC CONTENTS INTRODUCTION1 CITRIS AND2 MOTES 30 YEARS OF INNOVATION GENE MYERS4 Q&A 1973–20 0 3 6GRAPHICS INTELLIGENT SYSTEMS 1RESEARCH0 DEPARTMENT STATISTICS14 ROC-SOLID SYSTEMS16 USER INTERFACE DESIGN AND DEVELOPMENT20 INTERDISCIPLINARY THEOR22Y 30PROOF-CARRYING CODE 28 COMPLEXITY 30THEORY FACULTY32 THE COMPUTER SCIENCE DIVISION OF THE DEPARTMENT OF EECS AT UC BERKELEY WAS CREATED IN 1973. THIRTY YEARS OF INNOVATION IS A CELEBRATION OF THE ACHIEVEMENTS OF THE FACULTY, STAFF, STUDENTS AND ALUMNI DURING A PERIOD OF TRULY BREATHTAKING ADVANCES IN COMPUTER SCIENCE AND ENGINEERING. THE FIRST CHAIR OF COMPUTER research in theoretical computer science received a Turing Award in 1989 for this work. learning is bringing us ever closer to the dream SCIENCE AT BERKELEY was Richard Karp, at Berkeley. In the area of programming languages and of truly intelligent machines. who had just shown that the hardness of well- software engineering, Berkeley research has The impact of Berkeley research on the practi- Berkeley’s was the one of the first top comput- known algorithmic problems, such as finding been noted for its flair for combining theory cal end of computer science has been no less er science programs to invest seriously in com- the minimum cost tour for a travelling sales- and practice, as exemplified in these pages significant. The development of Reduced puter graphics, and our illustrious alumni in person, could be related to NP-completeness— by George Necula’s research on proof- Instruction Set computers by David Patterson that area have done us proud. We were not so a concept proposed earlier by former Berkeley carrying code.
    [Show full text]
  • Instructor Course Web Page Purpose Prerequisites Text Book
    CS 535 Advanced Operating Systems WPI, Fall 2013 Craig E. Wills Course Syllabus Monday, September 9, 2013 Instructor Craig E. Wills, FL-234, [email protected]. Office hours: as needed by students. Any time for short questions. Electronic mail is an effective method to contact me. Course Web Page Copies of all handouts, assignments and notes will be posted as appropriate on the course Web page. The address for it is http://www.cs.wpi.edu/~cs535/f13/. Purpose This is a graduate-level course in the design of advanced systems. It focuses on the issues of advanced operating systems and distributed systems, which have evolved from general- purpose multiprogramming systems covered in previous courses. The goals are 1) to famil- iarize the student with current literature in the area, 2) for students to be able to place work in its context in terms of its relative importance and relationship with other work, 3) to give students experience in making public presentations of technical work, and 4) to develop students’ ability for critical thinking and discussions concerning design choices, tradeoffs, and their consequences. Prerequisites A first course in operating systems, such as CS502. Background in architecture, networking, compilers and programming languages would also be helpful for issues related to these topics. An interest in reading, thinking about and discussing issues in advanced system design. Text Book There is no required text book for the course. Relevant text books are: 1. Distributed Systems Principles and Paradigms. Andrew S. Tanenbaum and Maarten van Steen, Prentice Hall, 2nd edition, 2007. 1 2.
    [Show full text]
  • Scalable Network Forensics
    Scalable Network Forensics Matthias Vallentin Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2016-55 http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-55.html May 12, 2016 Copyright © 2016, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Scalable Network Forensics by Matthias Vallentin A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Vern Paxson, Chair Professor Michael Franklin Professor David Brillinger Spring 2016 Scalable Network Forensics Copyright 2016 by Matthias Vallentin 1 Abstract Scalable Network Forensics by Matthias Vallentin Doctor of Philosophy in Computer Science University of California, Berkeley Professor Vern Paxson, Chair Network forensics and incident response play a vital role in site operations, but for large networks can pose daunting difficulties to cope with the ever-growing volume of activity and resulting logs. On the one hand, logging sources can generate tens of thousands of events
    [Show full text]
  • Contents U U U
    Contents u u u ACM Awards Reception and Banquet, June 2018 .................................................. 2 Introduction ......................................................................................................................... 3 A.M. Turing Award .............................................................................................................. 4 ACM Prize in Computing ................................................................................................. 5 ACM Charles P. “Chuck” Thacker Breakthrough in Computing Award ............. 6 ACM – AAAI Allen Newell Award .................................................................................. 7 Software System Award ................................................................................................... 8 Grace Murray Hopper Award ......................................................................................... 9 Paris Kanellakis Theory and Practice Award ...........................................................10 Karl V. Karlstrom Outstanding Educator Award .....................................................11 Eugene L. Lawler Award for Humanitarian Contributions within Computer Science and Informatics ..........................................................12 Distinguished Service Award .......................................................................................13 ACM Athena Lecturer Award ........................................................................................14 Outstanding Contribution
    [Show full text]
  • Redundant Disk Arrays: Reliable, Parallel Secondary Storage Garth Alan Gibson
    https://ntrs.nasa.gov/search.jsp?R=19920010046 2020-03-17T12:45:44+00:00Z Redundant Disk Arrays: Reliable, Parallel Secondary Storage Garth Alan Gibson (NASA-CR-189962) REDUNDANT DISK ARRAYS: N92-19288 RELIABLE, PARALLEL SECONDARY STORAGE Ph.D. Thesis (California Univ.) 267 p CSCL 09B Unclas G3/&0 0073849 Report No. UCB/CSD 91/613 December 1990 Computer Science Division (EECS) University of California, Berkeley Berkeley, California 94720 Redundant Disk Arrays: Reliable, Parallel Secondary Storage By Garth Alan Gibson B.Math.(Hons.) (University of Waterloo) 1983 M.S. (University of California) 1987 DISSERTATION Submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in COMPUTER SCIENCE in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA at BERKELEY ********************************** Redundant Disk Arrays: Reliable, Parallel Secondary Storage by Garth Alan Gibson Abstract During the past decade, advances in processor and memory technology have given rise to increases in computational performance that far outstrip increases in the performance of secon- dary storage technology. Coupled with emerging small-disk technology, disk arrays provide the cost, volume, and capacity of current disk subsystems but, by leveraging parallelism, many times their performance. Unfortunately, arrays of small disks may have much higher failure rates than the single large disks they replace. Redundant Arrays of Inexpensive Disks (RAID) use simple redundancy schemes to provide high data reliability. This dissertation investigates the data encoding, performance, and reliability of redundant disk arrays. Organizing redundant data into a disk array is treated as a coding problem in this disserta- tion. Among alternatives examined, codes as simple as parity are shown to effectively correct single, self-identifying disk failures.
    [Show full text]
  • Cloud Computing and Its Applications: a Comprehensive Survey
    IJCA, Vol. 28, No. 1, March 2021 3 Cloud Computing and Its Applications: A Comprehensive Survey Jalal H. Kiswani*, Sergiu M. Dascalu*, and Frederick C. Harris, Jr.* University of Nevada, Reno, USA. * Abstract Organizations would buy computers, install them locally on their premises, and use them for their computational needs. Cloud computing is one of the most significant trends in Even with the rise of personal computers in the 1980’s, the the information technology evolution, as it has created new same behavior continued, so that organizations and individuals opportunities that were never possible before. It is utilized followed the same approach. During those eras, some people and adopted by individuals and businesses on all scales, from a and organizations thought about a different approach: why not cloud-storage service such as Google Drive for normal users, to offer computation hardware and/or software as utility services, large scale integrated servers for online social media platforms same as electricity or telephone landlines? Thus, users could such as Facebook. In cloud computing, services are offered use these services on a subscription-based approach (e.g., such mainly on three levels: infrastructure, platform, and software. as monthly or yearly), and pay-as-they-use (i.e., usage-based In this article, an extensive and detailed literature review about costing) [88]. cloud computing and its applications is presented, including John McCarthy was the first to present the idea of offering history and evolution. Moreover, to measure the adoption of computing as utility in 1961 [25]. Five years later, “The cloud applications in industry and academia, we conducted a Challenge of the Computer Utility” book written by Parkhill user-study survey that included professionals and academics discussed many aspects of computing as a utility [70] .
    [Show full text]
  • THE DESIGN of XPRS Michael Stonebraker, Randy Katz, David
    THE DESIGN OF XPRS Michael Stonebraker, Randy Katz, David Patterson, and John Ousterhout EECS Dept. University of California, Berkeley Abstract This paper presents an overview of the techniques we are using to build a DBMS at Berkeley that will simultaneously provide high performance and high availability in transaction processing environments, in applications with complex ad-hoc queries and in applications with large objects such as images or CAD layouts. We plan to achieve these goals using a general purpose DBMS and operating system and a shared memory multiprocessor. The hardware and software tactics which we are using to accomplish these goals are described in this paper and include a novel “fast path” feature, a special purpose concurrency control scheme, a two-dimensional file system, exploitation of parallelism and a novel method to efficiently mirror disks. 1. INTRODUCTION At Berkeley we are constructing a high performance data base system with novel software and hard- ware assists. The basic goals of XPRS (eXtended Postgres on Raid and Sprite) are very high performance from a general purpose DBMS running on a conventional operating system and very high availability. Moreover, we plan to optimize for either a single CPU in a computer system (e.g. a Sun 4) or a shared memory multiprocessor (e.g a SEQUENT Symmetry system). We discuss each goal in turn in the remain- der of this introduction and then discuss why we hav echosen to exploit shared memory over shared nothing or shared disk. 1.1. High Performance We strive for high performance in three different application areas: 1) transaction processing 2) complex ad-hoc queries 3) management of large objects Previous high transaction rate systems have been built on top of low-level, hard-to-program, underlying data managers such as TPF [BAMB87] and IMS/Fast Path [DATE84].
    [Show full text]