Replication and Caching Systems for the Support of Vms Stored in File Systems with Snapshots
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Operating System Boot from Fully Encrypted Device
Masaryk University Faculty of Informatics Operating system boot from fully encrypted device Bachelor’s Thesis Daniel Chromik Brno, Fall 2016 Replace this page with a copy of the official signed thesis assignment and the copy of the Statement of an Author. Declaration Hereby I declare that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Daniel Chromik Advisor: ing. Milan Brož i Acknowledgement I would like to thank my advisor, Ing. Milan Brož, for his guidance and his patience of a saint. Another round of thanks I would like to send towards my family and friends for their support. ii Abstract The goal of this work is description of existing solutions for boot- ing Linux and Windows from fully encrypted devices with Secure Boot. Before that, though, early boot process and bootloaders are de- scribed. A simple Linux distribution is then set up to boot from a fully encrypted device. And lastly, existing Windows encryption solutions are described. iii Keywords boot process, Linux, Windows, disk encryption, GRUB 2, LUKS iv Contents 1 Introduction ............................1 1.1 Thesis goals ..........................1 1.2 Thesis structure ........................2 2 Boot Process Description ....................3 2.1 Early Boot Process ......................3 2.2 Firmware interfaces ......................4 2.2.1 BIOS – Basic Input/Output System . .4 2.2.2 UEFI – Unified Extended Firmware Interface .5 2.3 Partitioning tables ......................5 2.3.1 MBR – Master Boot Record . -
White Paper: Indestructible Firewall in a Box V1.0 Nick Mccubbins
White Paper: Indestructible Firewall In A Box v1.0 Nick McCubbins 1.1 Credits • Nathan Yawn ([email protected]) 1.2 Acknowledgements • Firewall-HOWTO • Linux Router Project • LEM 1.3 Revision History • Version 1.0 First public release 1.4 Feedback • Send all information and/or criticisms to [email protected] 1.5 Distribution Policy 2 Abstract In this document, the procedure for creating an embedded firewall whose root filesystem is loaded from a flash disk and then executed from a RAMdisk will be illustrated. A machine such as this has uses in many environments, from corporate internet access to sharing of a cable modem or xDSL connection among many computers. It has the advantages of being very light and fast, being impervious to filesystem corruption due to power loss, and being largely impervious to malicious crackers. The type of firewall illustrated herein is a simple packet-filtering, masquerading setup. Facilities for this already exist in the Linux kernel, keeping the system's memory footprint small. As such the device lends itself to embedding very well. For a more detailed description of firewall particulars, see the Linux Firewall-HOWTO. 3 Equipment This project has minimal hardware requirements. An excellent configuration consists of: For a 100-baseT network: • SBC-554 Pentium SBC with PISA bus and on-board PCI NIC (http://www.emacinc.com/pc.htm#pentiumsbc), approx. $373 • PISA backplane, chassis, power supply (http://www.emacinc.com/sbcpc_addons/mbpc641.htm), approx. $305 • Second PCI NIC • 32 MB RAM • 4 MB M-Systems Flash Disk (minimum), approx. $45 For a 10-baseT network: • EMAC's Standard Server-in-a-Box product (http://www.emacinc.com/server_in_a_box.htm), approx. -
Quantum Dxi4500 User's Guide
User’s Guide User’s Guide User’s Guide User’s Guide QuantumQuantum DXDXii45004500 DX i -Series 6-66904-01 Rev C DXi4500 User’s Guide, 6-66904-01 Rev C, August 2010, Product of USA. Quantum Corporation provides this publication “as is” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Quantum Corporation may revise this publication from time to time without notice. COPYRIGHT STATEMENT © 2010 Quantum Corporation. All rights reserved. Your right to copy this manual is limited by copyright law. Making copies or adaptations without prior written authorization of Quantum Corporation is prohibited by law and constitutes a punishable violation of the law. TRADEMARK STATEMENT Quantum, the Quantum logo, DLT, DLTtape, the DLTtape logo, Scalar, and StorNext are registered trademarks of Quantum Corporation, registered in the U.S. and other countries. Backup. Recovery. Archive. It’s What We Do., the DLT logo, DLTSage, DXi, DXi-Series, Dynamic Powerdown, FastSense, FlexLink, GoVault, MediaShield, Optyon, Pocket-sized. Well-armored, SDLT, SiteCare, SmartVerify, StorageCare, Super DLTtape, SuperLoader, and Vision are trademarks of Quantum. LTO and Ultrium are trademarks of HP, IBM, and Quantum in the U.S. and other countries. All other trademarks are the property of their respective companies. Specifications are subject to change without notice. ii Quantum DXi4500 User’s Guide Contents Preface xv Chapter 1 DXi4500 System Description 1 Overview . 1 Features and Benefits . 3 Data Reduction . 3 Data Deduplication . 3 Compression. 4 Space Reclamation . 4 Remote Replication . 5 DXi4500 System . -
Replication Using Group Communication Over a Partitioned Network
Replication Using Group Communication Over a Partitioned Network Thesis submitted for the degree “Doctor of Philosophy” Yair Amir Submitted to the Senate of the Hebrew University of Jerusalem (1995). This work was carried out under the supervision of Professor Danny Dolev ii Acknowledgments I am deeply grateful to Danny Dolev, my advisor and mentor. I thank Danny for believing in my research, for spending so many hours on it, and for giving it the theoretical touch. His warm support and patient guidance helped me through. I hope I managed to adopt some of his professional attitude and integrity. I thank Daila Malki for her help during the early stages of the Transis project. Thanks to Idit Keidar for helping me sharpen some of the issues of the replication server. I enjoyed my collaboration with Ofir Amir on developing the coloring model of the replication server. Many thanks to Roman Vitenberg for his valuable insights regarding the extended virtual synchrony model and the replication algorithm. I benefited a lot from many discussions with Ahmad Khalaila regarding distributed systems and other issues. My thanks go to David Breitgand, Gregory Chokler, Yair Gofen, Nabil Huleihel and Rimon Orni, for their contribution to the Transis project and to my research. I am grateful to Michael Melliar-Smith and Louise Moser from the Department of Electrical and Computer Engineering, University of California, Santa Barbara. During two summers, several mutual visits and extensive electronic correspondence, Louise and Michael were involved in almost every aspect of my research, and unofficially served as my co-advisors. The work with Deb Agarwal and Paul Ciarfella on the Totem protocol contributed a lot to my understanding of high-speed group communication. -
Chapter 3. Booting Operating Systems
Chapter 3. Booting Operating Systems Abstract: Chapter 3 provides a complete coverage on operating systems booting. It explains the booting principle and the booting sequence of various kinds of bootable devices. These include booting from floppy disk, hard disk, CDROM and USB drives. Instead of writing a customized booter to boot up only MTX, it shows how to develop booter programs to boot up real operating systems, such as Linux, from a variety of bootable devices. In particular, it shows how to boot up generic Linux bzImage kernels with initial ramdisk support. It is shown that the hard disk and CDROM booters developed in this book are comparable to GRUB and isolinux in performance. In addition, it demonstrates the booter programs by sample systems. 3.1. Booting Booting, which is short for bootstrap, refers to the process of loading an operating system image into computer memory and starting up the operating system. As such, it is the first step to run an operating system. Despite its importance and widespread interests among computer users, the subject of booting is rarely discussed in operating system books. Information on booting are usually scattered and, in most cases, incomplete. A systematic treatment of the booting process has been lacking. The purpose of this chapter is to try to fill this void. In this chapter, we shall discuss the booting principle and show how to write booter programs to boot up real operating systems. As one might expect, the booting process is highly machine dependent. To be more specific, we shall only consider the booting process of Intel x86 based PCs. -
Replication Vs. Caching Strategies for Distributed Metadata Management in Cloud Storage Systems
ShardFS vs. IndexFS: Replication vs. Caching Strategies for Distributed Metadata Management in Cloud Storage Systems Lin Xiao, Kai Ren, Qing Zheng, Garth A. Gibson Carnegie Mellon University {lxiao, kair, zhengq, garth}@cs.cmu.edu Abstract 1. Introduction The rapid growth of cloud storage systems calls for fast and Modern distributed storage systems commonly use an archi- scalable namespace processing. While few commercial file tecture that decouples metadata access from file read and systems offer anything better than federating individually write operations [21, 25, 46]. In such a file system, clients non-scalable namespace servers, a recent academic file sys- acquire permission and file location information from meta- tem, IndexFS, demonstrates scalable namespace processing data servers before accessing file contents, so slower meta- based on client caching of directory entries and permissions data service can degrade performance for the data path. Pop- (directory lookup state) with no per-client state in servers. In ular new distributed file systems such as HDFS [25] and the this paper we explore explicit replication of directory lookup first generation of the Google file system [21] have used cen- state in all servers as an alternative to caching this infor- tralized single-node metadata services and focused on scal- mation in all clients. Both eliminate most repeated RPCs to ing only the data path. However, the single-node metadata different servers in order to resolve hierarchical permission server design limits the scalability of the file system in terms tests. Our realization for server replicated directory lookup of the number of objects stored and concurrent accesses to state, ShardFS, employs a novel file system specific hybrid the file system [40]. -
Replication As a Prelude to Erasure Coding in Data-Intensive Scalable Computing
DiskReduce: Replication as a Prelude to Erasure Coding in Data-Intensive Scalable Computing Bin Fan Wittawat Tantisiriroj Lin Xiao Garth Gibson fbinfan , wtantisi , lxiao , garth.gibsong @ cs.cmu.edu CMU-PDL-11-112 October 2011 Parallel Data Laboratory Carnegie Mellon University Pittsburgh, PA 15213-3890 Acknowledgements: We would like to thank several people who made significant contributions. Robert Chansler, Raj Merchia and Hong Tang from Yahoo! provide various help. Dhruba Borthakur from Facebook provides statistics and feedback. Bin Fu and Brendan Meeder gave us their scientific applications and data-sets for experimental evaluation. The work in this paper is based on research supported in part by the Betty and Gordon Moore Foundation, by the Department of Energy, under award number DE-FC02-06ER25767, by the Los Alamos National Laboratory, by the Qatar National Research Fund, under award number NPRP 09-1116-1-172, under contract number 54515- 001-07, by the National Science Foundation under awards CCF-1019104, CCF-0964474, OCI-0852543, CNS-0546551 and SCI-0430781, and by Google and Yahoo! research awards. We also thank the members and companies of the PDL Consortium (including APC, EMC, Facebook, Google, Hewlett-Packard, Hitachi, IBM, Intel, Microsoft, NEC, NetApp, Oracle, Panasas, Riverbed, Samsung, Seagate, STEC, Symantec, and VMware) for their interest, insights, feedback, and support. Keywords: Distributed File System, RAID, Cloud Computing, DISC Abstract The first generation of Data-Intensive Scalable Computing file systems such as Google File System and Hadoop Distributed File System employed n (n ≥ 3) replications for high data reliability, therefore delivering users only about 1=n of the total storage capacity of the raw disks. -
A Lightweight Virtualization Layer with Hardware-Assistance for Embedded Systems
PONTIFICAL CATHOLIC UNIVERSITY OF RIO GRANDE DO SUL FACULTY OF INFORMATICS COMPUTER SCIENCE GRADUATE PROGRAM A LIGHTWEIGHT VIRTUALIZATION LAYER WITH HARDWARE-ASSISTANCE FOR EMBEDDED SYSTEMS CARLOS ROBERTO MORATELLI Dissertation submitted to the Pontifical Catholic University of Rio Grande do Sul in partial fullfillment of the requirements for the degree of Ph. D. in Computer Science. Advisor: Prof. Fabiano Hessel Porto Alegre 2016 To my family and friends. “I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones.” (Linus Torvalds) ACKNOWLEDGMENTS I would like to express my sincere gratitude to those who helped me throughout all my Ph.D. years and made this dissertation possible. First of all, I would like to thank my advisor, Prof. Fabiano Passuelo Hessel, who has given me the opportunity to undertake a Ph.D. and provided me invaluable guidance and support in my Ph.D. and in my academic life in general. Thank you to all the Ph.D. committee members – Prof. Carlos Eduardo Pereira (dissertation proposal), Prof. Rodolfo Jardim de Azevedo, Prof. Rômulo Silva de Oliveira and Prof. Tiago Ferreto - for the time invested and for the valuable feedback provided.Thank you Dr. Luca Carloni and the other SLD team members at the Columbia University in the City of New York for receiving me and giving me the opportunity to work with them during my ‘sandwich’ research internship. Eu gostaria de agraceder minha esposa, Ana Claudia, por ter estado ao meu lado durante todo o meu período de doutorado. O seu apoio e compreensão foram e continuam sendo muito importantes para mim. -
Network Boot and Exotic Root HOWTO
Network Boot and Exotic Root HOWTO Brieuc Jeunhomme frtest [email protected] Logilab S.A. Revision History Revision 0.3 2002−04−28 Revised by: bej Many feedback inclusions, added links to several projects Revision 0.2.2 2001−12−08 Revised by: dcm Licensed GFDL Revision 0.2.1 2001−05−21 Revised by: logilab Fixed bibliography and artheader Revision 0.2 2001−05−19 Revised by: bej Many improvements and included Ken Yap's feedback. Revision 0.1.1 2001−04−09 Revised by: logilab First public draft. Revision 0.1 2000−12−09 Revised by: bej Initial draft. This document explains how to quickly setup a linux server to provide what diskless linux clients require to get up and running, using an IP network. It includes data and partly rewritten text from the Diskless−HOWTO, the Diskless−root−NFS−HOWTO, the linux kernel documentation, the etherboot project's documentation, the linux terminal server project's homepage, and the author's personal experience, acquired when working for Logilab. Eventually this document may end up deprecating the Diskless−HOWTO and Diskless−root−NFS−HOWTO. Please note that you'll also find useful information in the From−PowerUp−to−bash−prompt−HOWTO and the Thin−Client−HOWTO, and the Claus−Justus Heine's page about NFS swapping. Network Boot and Exotic Root HOWTO Table of Contents 1. Introduction.....................................................................................................................................................1 1.1. What is this all about?.......................................................................................................................1 1.2. Thanks...............................................................................................................................................1 1.3. Diskless booting advocacy................................................................................................................1 1.3.1. Buying is cheaper than building.......................................................................................1 1.3.2. -
Middleware-Based Database Replication: the Gaps Between Theory and Practice
Appears in Proceedings of the ACM SIGMOD Conference, Vancouver, Canada (June 2008) Middleware-based Database Replication: The Gaps Between Theory and Practice Emmanuel Cecchet George Candea Anastasia Ailamaki EPFL EPFL & Aster Data Systems EPFL & Carnegie Mellon University Lausanne, Switzerland Lausanne, Switzerland Lausanne, Switzerland [email protected] [email protected] [email protected] ABSTRACT There exist replication “solutions” for every major DBMS, from Oracle RAC™, Streams™ and DataGuard™ to Slony-I for The need for high availability and performance in data Postgres, MySQL replication and cluster, and everything in- management systems has been fueling a long running interest in between. The naïve observer may conclude that such variety of database replication from both academia and industry. However, replication systems indicates a solved problem; the reality, academic groups often attack replication problems in isolation, however, is the exact opposite. Replication still falls short of overlooking the need for completeness in their solutions, while customer expectations, which explains the continued interest in developing new approaches, resulting in a dazzling variety of commercial teams take a holistic approach that often misses offerings. opportunities for fundamental innovation. This has created over time a gap between academic research and industrial practice. Even the “simple” cases are challenging at large scale. We deployed a replication system for a large travel ticket brokering This paper aims to characterize the gap along three axes: system at a Fortune-500 company faced with a workload where performance, availability, and administration. We build on our 95% of transactions were read-only. Still, the 5% write workload own experience developing and deploying replication systems in resulted in thousands of update requests per second, which commercial and academic settings, as well as on a large body of implied that a system using 2-phase-commit, or any other form of prior related work. -
Manufacturing Equipment Technologies for Hard Disk's
Manufacturing Equipment Technologies for Hard Disk’s Challenge of Physical Limits 222 Manufacturing Equipment Technologies for Hard Disk’s Challenge of Physical Limits Kyoichi Mori OVERVIEW: To meet the world’s growing demands for volume information, Brian Rattray not just with computers but digitalization and video etc. the HDD must Yuichi Matsui, Dr. Eng. continually increase its capacity and meet expectations for reduced power consumption and green IT. Up until now the HDD has undergone many innovative technological developments to achieve higher recording densities. To continue this increase, innovative new technology is again required and is currently being developed at a brisk pace. The key components for areal density improvements, the disk and head, require high levels of performance and reliability from production and inspection equipment for efficient manufacturing and stable quality assurance. To meet this demand, high frequency electronics, servo positioning and optical inspection technology is being developed and equipment provided. Hitachi High-Technologies Corporation is doing its part to meet market needs for increased production and the adoption of next-generation technology by developing the technology and providing disk and head manufacturing/inspection equipment (see Fig. 1). INTRODUCTION higher efficiency storage, namely higher density HDDS (hard disk drives) have long relied on the HDDs will play a major role. computer market for growth but in recent years To increase density, the performance and quality of there has been a shift towards cloud computing and the HDD’s key components, disks (media) and heads consumer electronics etc. along with a rapid expansion have most effect. Therefore further technological of data storage applications (see Fig. -
Sbadmin for Linux System Recovery Guide Is a Supplement to the Sbadmin User Guide, Providing Details on Reinstalling a Linux System from a Sbadmin System Backup
Linux System Recovery Guide Version 8.2 Trademarks and Copyrights © Copyright Storix, Inc. 1999-2021 SBAdmin is a registered trademark of Storix, Inc. SBAdmin is a trademark of Storix, Inc in the USA and other countries Intel is a registered trademark of Intel, Inc. Linux is a registered trademark of Linus Torvalds. Intel, Pentium, IA32, Itanium, Celeron and IA64 are registered trademarks of Intel Corporation. AMD, Opteron, and Athlon are registered trademarks of Advanced Micro Devices. HP Integrity servers are registered trademarks of Hewlett-Packard Development Company. Publicly Available Software This product either includes or is developed using source code that is publicly available: AESCrypt* Rijndael and Cipher Block Feedback Copyright 1999, 2000 Enhanced Software Technologies Inc. mode (CFB-128) encryption/decryption http://aescrypt.sourceforge.net/ algorithms BusyBox Single executable containing tiny Copyright 1989, 1991 Free Software Foundation, Inc. versions of common UNIX utilities http://busybox.net/cgi-bin/cvsweb/busybox/ LILO LInux boot Loader Copyright 1999-2003 John Coffman. Copyright 1992-1998 Werner Almesberger. http://freshmeat.net/projects/lilo/ Tcl Open source scripting language Copyright Regents of the University of California, Sun Microsystems, Inc. http://tcl.sourceforge.net Tk Tk graphics toolkit Copyright Regents of the University of California, Sun Microsystems, Inc. http://tcl.sourceforge.net DropBear A Smallish SSH 2 Server and Client Copyright 2002, 2003 Matt Johnston http://www.matt.ucc.asn.au/dropbear/dropbear.html GRUB Grand Unified Bootloader (GNU GRUB) Copyright 1989, 1991 Free Software Foundation, Inc. http://www.gnu.org/software/grub/grub.html Lighttpd Secure, fast, compliant and flexible Copyright 2004 Jan Kneschkle, incremental web-server http://www.lighttpd.net OpenSSL Toolkit implementing Secure Socket Copyright 1998-2008 The OpenSSL Project Layer Copyright 1995-1998 Eric A.