A Study of Cryptographic File Systems in Userspace
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
The Google File System (GFS)
The Google File System (GFS), as described by Ghemwat, Gobioff, and Leung in 2003, provided the architecture for scalable, fault-tolerant data management within the context of Google. These architectural choices, however, resulted in sub-optimal performance in regard to data trustworthiness (security) and simplicity. Additionally, the application- specific nature of GFS limits the general scalability of the system outside of the specific design considerations within the context of Google. SYSTEM DESIGN: The authors enumerate their specific considerations as: (1) commodity components with high expectation of failure, (2) a system optimized to handle relatively large files, particularly multi-GB files, (3) most writes to the system are concurrent append operations, rather than internally modifying the extant files, and (4) a high rate of sustained bandwidth (Section 1, 2.1). Utilizing these considerations, this paper analyzes the success of GFS’s design in achieving a fault-tolerant, scalable system while also considering the faults of the system with regards to data-trustworthiness and simplicity. Data-trustworthiness: In designing the GFS system, the authors made the conscious decision not prioritize data trustworthiness by allowing applications to access ‘stale’, or not up-to-date, data. In particular, although the system does inform applications of the chunk-server version number, the designers of GFS encouraged user applications to cache chunkserver information, allowing stale data accesses (Section 2.7.2, 4.5). Although possibly troubling -
A Review on GOOGLE File System
International Journal of Computer Science Trends and Technology (IJCST) – Volume 4 Issue 4, Jul - Aug 2016 RESEARCH ARTICLE OPEN ACCESS A Review on Google File System Richa Pandey [1], S.P Sah [2] Department of Computer Science Graphic Era Hill University Uttarakhand – India ABSTRACT Google is an American multinational technology company specializing in Internet-related services and products. A Google file system help in managing the large amount of data which is spread in various databases. A good Google file system is that which have the capability to handle the fault, the replication of data, make the data efficient, memory storage. The large data which is big data must be in the form that it can be managed easily. Many applications like Gmail, Facebook etc. have the file systems which organize the data in relevant format. In conclusion the paper introduces new approaches in distributed file system like spreading file’s data across storage, single master and appends writes etc. Keywords:- GFS, NFS, AFS, HDFS I. INTRODUCTION III. KEY IDEAS The Google file system is designed in such a way that 3.1. Design and Architecture: GFS cluster consist the data which is spread in database must be saved in of single master and multiple chunk servers used by a arrange manner. multiple clients. Since files to be stored in GFS are The arrangement of data is in the way that it can large, processing and transferring such huge files can reduce the overhead load on the server, the consume a lot of bandwidth. To efficiently utilize availability must be increased, throughput should be bandwidth files are divided into large 64 MB size highly aggregated and much more services to make chunks which are identified by unique 64-bit chunk the available data more accurate and reliable. -
F1 Query: Declarative Querying at Scale
F1 Query: Declarative Querying at Scale Bart Samwel John Cieslewicz Ben Handy Jason Govig Petros Venetis Chanjun Yang Keith Peters Jeff Shute Daniel Tenedorio Himani Apte Felix Weigel David Wilhite Jiacheng Yang Jun Xu Jiexing Li Zhan Yuan Craig Chasseur Qiang Zeng Ian Rae Anurag Biyani Andrew Harn Yang Xia Andrey Gubichev Amr El-Helw Orri Erling Zhepeng Yan Mohan Yang Yiqun Wei Thanh Do Colin Zheng Goetz Graefe Somayeh Sardashti Ahmed M. Aly Divy Agrawal Ashish Gupta Shiv Venkataraman Google LLC [email protected] ABSTRACT 1. INTRODUCTION F1 Query is a stand-alone, federated query processing platform The data processing and analysis use cases in large organiza- that executes SQL queries against data stored in different file- tions like Google exhibit diverse requirements in data sizes, la- based formats as well as different storage systems at Google (e.g., tency, data sources and sinks, freshness, and the need for custom Bigtable, Spanner, Google Spreadsheets, etc.). F1 Query elimi- business logic. As a result, many data processing systems focus on nates the need to maintain the traditional distinction between dif- a particular slice of this requirements space, for instance on either ferent types of data processing workloads by simultaneously sup- transactional-style queries, medium-sized OLAP queries, or huge porting: (i) OLTP-style point queries that affect only a few records; Extract-Transform-Load (ETL) pipelines. Some systems are highly (ii) low-latency OLAP querying of large amounts of data; and (iii) extensible, while others are not. Some systems function mostly as a large ETL pipelines. F1 Query has also significantly reduced the closed silo, while others can easily pull in data from other sources. -
Cloud Computing Bible Is a Wide-Ranging and Complete Reference
A thorough, down-to-earth look Barrie Sosinsky Cloud Computing Barrie Sosinsky is a veteran computer book writer at cloud computing specializing in network systems, databases, design, development, The chance to lower IT costs makes cloud computing a and testing. Among his 35 technical books have been Wiley’s Networking hot topic, and it’s getting hotter all the time. If you want Bible and many others on operating a terra firma take on everything you should know about systems, Web topics, storage, and the cloud, this book is it. Starting with a clear definition of application software. He has written nearly 500 articles for computer what cloud computing is, why it is, and its pros and cons, magazines and Web sites. Cloud Cloud Computing Bible is a wide-ranging and complete reference. You’ll get thoroughly up to speed on cloud platforms, infrastructure, services and applications, security, and much more. Computing • Learn what cloud computing is and what it is not • Assess the value of cloud computing, including licensing models, ROI, and more • Understand abstraction, partitioning, virtualization, capacity planning, and various programming solutions • See how to use Google®, Amazon®, and Microsoft® Web services effectively ® ™ • Explore cloud communication methods — IM, Twitter , Google Buzz , Explore the cloud with Facebook®, and others • Discover how cloud services are changing mobile phones — and vice versa this complete guide Understand all platforms and technologies www.wiley.com/compbooks Shelving Category: Use Google, Amazon, or -
Mapreduce: Simplified Data Processing On
MapReduce: Simplified Data Processing on Large Clusters Jeffrey Dean and Sanjay Ghemawat [email protected], [email protected] Google, Inc. Abstract given day, etc. Most such computations are conceptu- ally straightforward. However, the input data is usually MapReduce is a programming model and an associ- large and the computations have to be distributed across ated implementation for processing and generating large hundreds or thousands of machines in order to finish in data sets. Users specify a map function that processes a a reasonable amount of time. The issues of how to par- key/value pair to generate a set of intermediate key/value allelize the computation, distribute the data, and handle pairs, and a reduce function that merges all intermediate failures conspire to obscure the original simple compu- values associated with the same intermediate key. Many tation with large amounts of complex code to deal with real world tasks are expressible in this model, as shown these issues. in the paper. As a reaction to this complexity, we designed a new Programs written in this functional style are automati- abstraction that allows us to express the simple computa- cally parallelized and executed on a large cluster of com- tions we were trying to perform but hides the messy de- modity machines. The run-time system takes care of the tails of parallelization, fault-tolerance, data distribution details of partitioning the input data, scheduling the pro- and load balancing in a library. Our abstraction is in- gram's execution across a set of machines, handling ma- spired by the map and reduce primitives present in Lisp chine failures, and managing the required inter-machine and many other functional languages. -
Google File System 2.0: a Modern Design and Implementation
Google File System 2.0: A Modern Design and Implementation Babatunde Micheal Okutubo Gan Tu Xi Cheng Stanford University Stanford University Stanford University [email protected] [email protected] [email protected] Abstract build a GFS-2.0 from scratch, with modern software design principles and implementation patterns, in order to survey The Google File System (GFS) presented an elegant de- ways to not only effectively achieve various system require- sign and has shown tremendous capability to scale and to ments, but also with great support for high-concurrency tolerate fault. However, there is no concrete implementa- workloads and software extensibility. tion of GFS being open sourced, even after nearly 20 years A second motivation of this project is that GFS has a of its initial development. In this project, we attempt to build single master, which served well back in 2003 when this a GFS-2.0 in C++ from scratch1 by applying modern soft- work was published. However, as data volume, network ware design discipline. A fully functional GFS has been de- capacity as well as hardware capability all have been dras- veloped to support concurrent file creation, read and paral- tically improved from that date, our hypothesis is that a lel writes. The system is capable of surviving chunk servers multi-master version of GFS is advantageous as it provides going down during both read and write operations. Various better fault tolerance (especially from the master servers’ crucial implementation details have been discussed, and a point of view), availability and throughput. Although the micro-benchmark has been conducted and presented. -
Lightweight Virtualization with Gobolinux' Runner
Lightweight virtualization with GoboLinux’ Runner Lucas C. Villa Real [email protected] About GoboLinux ● Alternative distribution born in 2002 ● Explores novel ideas in the Linux distribution ecosystem ● Introduces a rather diferent directory hierarchy How diferent? lucasvr@fedora ~] ls / bin dev home lib64 media opt root sbin sys usr boot etc lib lost+found mnt proc run srv tmp var lucasvr@fedora ~] ls /usr bin games include lib lib64 libexec local sbin share src tmp lucasvr@fedora ~] ls /usr/local bin etc games include lib lib64 libexec sbin share src lucasvr@gobolinux ~] ls / Data Mount Programs System Users GoboLinux File System Hierarchy /Programs Self-contained programs: no need for a package manager ~] ls /Programs AbsTk DifUtils GnuTLS Kerberos LibXML2 ACL Dit GoboHide Kmod LibXSLT Acpid DosFSTools GParted Lame Linux AGNClient E2FSProgs Gperf LCMS Linux-Firmware ALSA-Lib EFIBootMgr GPM Less Linux-PAM ALSA-Utils ELFUtils Grep LibDRM Lsof APR EncFS Grof LibEvdev Lua APR-Util ExFAT GRUB LibExif LuaRocks … /Programs Multiple versions of a given program can coexist ~] ls /Programs/GTK+ 2.24.22 2.24.30 3.10.6 3.21.4 Current Settings ~] ls /Programs/GTK+/2.24.22 bin doc include lib Resources share ~] ls /Programs/GTK+/2.24.22/bin gtk-builder-convert gtk-demo gtk-query-immodules2.0 gtk-update-icon-cache ~] ls /Programs/GTK+/2.24.30/bin gtk-builder-convert gtk-demo gtk-query-immodules2.0 gtk-update-icon-cache /Programs Easy to tell which fles belongs to which packages lucasvr@fedora ~] ls -l /bin/bash -rwxr-xr-x. 1 root root 1072008 -
Operating System Support for Run-Time Security with a Trusted Execution Environment
Operating System Support for Run-Time Security with a Trusted Execution Environment - Usage Control and Trusted Storage for Linux-based Systems - by Javier Gonz´alez Ph.D Thesis IT University of Copenhagen Advisor: Philippe Bonnet Submitted: January 31, 2015 Last Revision: May 30, 2015 ITU DS-nummer: D-2015-107 ISSN: 1602-3536 ISBN: 978-87-7949-302-5 1 Contents Preface8 1 Introduction 10 1.1 Context....................................... 10 1.2 Problem....................................... 12 1.3 Approach...................................... 14 1.4 Contribution.................................... 15 1.5 Thesis Structure.................................. 16 I State of the Art 18 2 Trusted Execution Environments 20 2.1 Smart Cards.................................... 21 2.1.1 Secure Element............................... 23 2.2 Trusted Platform Module (TPM)......................... 23 2.3 Intel Security Extensions.............................. 26 2.3.1 Intel TXT.................................. 26 2.3.2 Intel SGX.................................. 27 2.4 ARM TrustZone.................................. 29 2.5 Other Techniques.................................. 32 2.5.1 Hardware Replication........................... 32 2.5.2 Hardware Virtualization.......................... 33 2.5.3 Only Software............................... 33 2.6 Discussion...................................... 33 3 Run-Time Security 36 3.1 Access and Usage Control............................. 36 3.2 Data Protection................................... 39 3.3 Reference -
A Survey on Cloud Storage
1764 JOURNAL OF COMPUTERS, VOL. 6, NO. 8, AUGUST 2011 A Survey on Cloud Storage Jiehui JU1,4 1.School of Information and Electronic Engineering, Zhejiang University of Science and Technology,Hangzhou,China Email: [email protected] Jiyi WU2,3, Jianqing FU3, Zhijie LIN1,3, Jianlin ZHANG2 2. Key Lab of E-Business and Information Security, Hangzhou Normal University, Hangzhou,China 3.School of Computer Science and Technology, Zhejiang University,Hangzhou,China 4.Key Lab of E-business Market Application Technology of Guangdong Province, Guangdong University of Business Studies, Guangzhou 510320, China Email: [email protected]; [email protected]; [email protected]; [email protected] Abstract— Cloud storage is a new concept come into being simultaneously with cloud computing, and can be divided into public cloud storage, private cloud storage and hybrid cloud storage. This article gives a quick introduction to cloud storage. It covers the key technologies in Cloud Computing and Cloud Storage. Google GFS massive data storage system and the popular open source Hadoop HDFS were detailed introduced to analyze the principle of Cloud Storage technology. As an important technology area and research direction, cloud storage is becoming a hot research for both academia and industry session. The future valuable research works were summarized at the end. Index Terms—Cloud Storage, Distributed File System, Research Status, survey I. INTRODUCTION As we all known disk storage is one of the largest expenditure in IT projects. ComputerWorld estimates that storage is responsible for almost 30% of capital expenditures as the average growth of data approaches Figure 1. -
Comparative Analysis of Distributed and Parallel File Systems' Internal Techniques
Comparative Analysis of Distributed and Parallel File Systems’ Internal Techniques Viacheslav Dubeyko Content 1 TERMINOLOGY AND ABBREVIATIONS ................................................................................ 4 2 INTRODUCTION......................................................................................................................... 5 3 COMPARATIVE ANALYSIS METHODOLOGY ....................................................................... 5 4 FILE SYSTEM FEATURES CLASSIFICATION ........................................................................ 5 4.1 Distributed File Systems ............................................................................................................................ 6 4.1.1 HDFS ..................................................................................................................................................... 6 4.1.2 GFS (Google File System) ....................................................................................................................... 7 4.1.3 InterMezzo ............................................................................................................................................ 9 4.1.4 CodA .................................................................................................................................................... 10 4.1.5 Ceph.................................................................................................................................................... 12 4.1.6 DDFS .................................................................................................................................................. -
A Novel Cryptographic Framework for Cloud File Systems and Cryfs, a Provably-Secure Construction
A Novel Cryptographic Framework for Cloud File Systems and CryFS, a Provably-Secure Construction Sebastian Messmer1, Jochen Rill2, Dirk Achenbach2, and J¨ornM¨uller-Quade3 1 [email protected] 2 FZI Forschungszentrum Informatik frill,[email protected] 3 Karlsruhe Institute of Technology (KIT) [email protected] Abstract. Using the cloud to store data offers many advantages for businesses and individuals alike. The cloud storage provider, however, has to be trusted not to inspect or even modify the data they are entrusted with. Encrypting the data offers a remedy, but current solutions have various drawbacks. Providers which offer encrypted storage themselves cannot necessarily be trusted, since they have no open implementation. Existing encrypted file systems are not designed for usage in the cloud and do not hide metadata like file sizes or directory structure, do not provide integrity, or are prohibitively inefficient. Most have no formal proof of security. Our contribution is twofold. We first introduce a comprehensive formal model for the security and integrity of cloud file systems. Second, we present CryFS, a novel encrypted file system specifically designed for usage in the cloud. Our file system protects confidentiality and integrity (including metadata), even in presence of an actively malicious cloud provider. We give a proof of security for these properties. Our implemen- tation is easy and transparent to use and offers performance comparable to other state-of-the-art file systems. 1 Introduction In recent years, cloud computing has transformed from a trend to a serious competition for traditional on-premise solutions. Elastic cost models and the availability of virtually infinite resources present an alternative to offers of a preset volume. -
Encfs Goes Multi-User: Adding Access Control to an Encrypted File System
c 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. http://ieeexplore.ieee.org/document/7860544/ EncFS goes Multi-User: Adding Access Control to an Encrypted File System Dominik Leibenger Jonas Fortmann Christoph Sorge CISPA, Saarland University University of Paderborn CISPA, Saarland University [email protected] [email protected] [email protected] Abstract—Among the different existing cryptographic file entities, which especially preserves the opportunity of creating systems, EncFS has a unique feature that makes it attractive for efficient, server-side snapshots if supported by the provider.1 backup setups involving untrusted (cloud) storage. It is a file- based overlay file system in normal operation (i.e., it maintains In contrast to other file-based encryption tools, EncFS a directory hierarchy by storing encrypted representations of has a unique feature: It allows to reverse its functionality files and folders in a specific source folder), but its reverse mode as to generate a deterministic, encrypted view of an existing allows to reverse this process: Users can mount deterministic, (unencrypted) folder on a local file system on the fly. The encrypted views of their local, unencrypted files on the fly, encrypted view can be synchronized to external, untrusted allowing synchronization to untrusted storage using standard cloud storage using standard tools like rsync [6] without hav- tools like rsync without having to store encrypted representations ing to store a local copy and without requiring changes to the on the local hard drive.