Newest Trends in High Performance File Systems
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
HP Storageworks Clustered File System Command Line Reference
HP StorageWorks Clustered File System 3.0 Command Line reference guide *392372-001* *392372–001* Part number: 392372–001 First edition: May 2005 Legal and notice information © Copyright 1999-2005 PolyServe, Inc. Portions © 2005 Hewlett-Packard Development Company, L.P. Neither PolyServe, Inc. nor Hewlett-Packard Company makes any warranty of any kind with regard to this material, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Neither PolyServe nor Hewlett-Packard shall be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this material. This document contains proprietary information, which is protected by copyright. No part of this document may be photocopied, reproduced, or translated into another language without the prior written consent of Hewlett-Packard. The information is provided “as is” without warranty of any kind and is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Neither PolyServe nor HP shall be liable for technical or editorial errors or omissions contained herein. The software this document describes is PolyServe confidential and proprietary. PolyServe and the PolyServe logo are trademarks of PolyServe, Inc. PolyServe Matrix Server contains software covered by the following copyrights and subject to the licenses included in the file thirdpartylicense.pdf, which is included in the PolyServe Matrix Server distribution. Copyright © 1999-2004, The Apache Software Foundation. Copyright © 1992, 1993 Simmule Turner and Rich Salz. -
Truenas® Privacy and Security Compliance Features
TRUENAS® PRIVACY AND SECURITY COMPLIANCE FEATURES Risk accountability EPR HIPAA information users PCI DSS ZFS HITECH TrueNAS ePHI branches corporate EPH health internal storage Compliance FreeNAS external process Audit encryption management patient GUI GDPRBackup data GRCFIPS 140-2 FreeBSD technology Governance enterprise NO MATTER ITS SIZE, EVERY BUSINESS TRUENAS PROVIDES FEATURES FOR REAL OPERATES IN A REGULATED ENVIRONMENT SECURITY AND COMPLIANCE Thanks to legislation like the European Union General TrueNAS is a unified file, block and object storage Data Protection Regulation (GDPR), it’s no longer only solution built on the OpenZFS self-healing file system government and medical providers that need to comply that supports hybrid and all-flash configurations. Unlike with strict privacy and security regulations. If your many competing storage systems, each TrueNAS scales business handles credit card information or customer from a few workgroup terabytes to multiple private personal information, you must navigate an alphabet cloud petabytes, all with a common user experience and soup of regulations that each include distinct obligations full data interoperability. and equally-distinct penalties for failing to comply with those obligations. From PCI DSS to the GDPR to TrueNAS uses a myriad of network and storage HIPAA, a common theme of data security stands out as encryption techniques to safeguard your data a fundamental requirement for regulation compliance throughout its life cycle and help assure your regulation and TrueNAS is ready -
Shared File Systems: Determining the Best Choice for Your Distributed SAS® Foundation Applications Margaret Crevar, SAS Institute Inc., Cary, NC
Paper SAS569-2017 Shared File Systems: Determining the Best Choice for your Distributed SAS® Foundation Applications Margaret Crevar, SAS Institute Inc., Cary, NC ABSTRACT If you are planning on deploying SAS® Grid Manager and SAS® Enterprise BI (or other distributed SAS® Foundation applications) with load balanced servers on multiple operating systems instances, , a shared file system is required. In order to determine the best shared file system choice for a given deployment, it is important to understand how the file system is used, the SAS® I/O workload characteristics performed on it, and the stressors that SAS Foundation applications produce on the file system. For the purposes of this paper, we use the term "shared file system" to mean both a clustered file system and shared file system, even though" shared" can denote a network file system and a distributed file system – not clustered. INTRODUCTION This paper examines the shared file systems that are most commonly used with SAS and reviews their strengths and weaknesses. SAS GRID COMPUTING REQUIREMENTS FOR SHARED FILE SYSTEMS Before we get into the reasons why a shared file system is needed for SAS® Grid Computing, let’s briefly discuss the SAS I/O characteristics. GENERAL SAS I/O CHARACTERISTICS SAS Foundation creates a high volume of predominately large-block, sequential access I/O, generally at block sizes of 64K, 128K, or 256K, and the interactions with data storage are significantly different from typical interactive applications and RDBMSs. Here are some major points to understand (more details about the bullets below can be found in this paper): SAS tends to perform large sequential Reads and Writes. -
Ed 377 034 Title Institution Pub Date Note Available
DOCUMENT RESUME ED 377 034 SE 054 362 TITLE Education & Recycling: Educator's Waste Management Resource and Activity Guide 1994. INSTITUTION California State Dept. of Conservation. Sacramento. Div. of Recycling. PUB DATE 94 NOTE 234p. AVAILABLE FROMCalifornia Department of Conservation, Division of Recycling, 801 K Street, MS 22-57, Sacramento, CA 95814. PUB TYPE Guides Classroom Use Teaching Guides (For Teacher) (052) EDRS PRICE MF01/PC10 Plus Postage. DESCRIPTORS Bilingual Instructional Materials; *Class Activities; Constructivism (Learning); *Educational Resources; Elementary Secondary Education; *Environmental Education; Evaluation Methods; *Recycling; Solid Wastes; Teaching Guides; *Waste Disposal; Worksheets IDENTIFIERS *California ABSTRACT This activity guide for grades K-12 reinforces the concepts of recycling, reducing, and reusing through a series of youth-oriented activities. The guide incorporates a video-based activity, multiple session classroom activities, and activities requiring group participation and student conducted research. Constructivist learning theory was considered during the development of activities. The guide is divided into the following sections:(1) 12 elementary and mieldle school classroom activities;(2) eight middle and high school classroom activities;(3) school recycling programs;(4) trivia, facts, and other information;(5) listing of 338 supplementary materials (activity, booklets, coloring and comic books, books, catalogs, curricula, extras, magazines, recycling programs, and videos);(6) listing of 39 environmental organizations; (7) approximately 1,300 California local government and community contacts; and (8)a glossary. Many activities incorporate science, history and social science, English and languege arts, and mathematics and art. Most activities include methods for teacher and student evaluations. Spanish translations are provided for some activity materials, including letters to parents, several take-home activities and the glossary. -
Comparative Analysis of Distributed and Parallel File Systems' Internal Techniques
Comparative Analysis of Distributed and Parallel File Systems’ Internal Techniques Viacheslav Dubeyko Content 1 TERMINOLOGY AND ABBREVIATIONS ................................................................................ 4 2 INTRODUCTION......................................................................................................................... 5 3 COMPARATIVE ANALYSIS METHODOLOGY ....................................................................... 5 4 FILE SYSTEM FEATURES CLASSIFICATION ........................................................................ 5 4.1 Distributed File Systems ............................................................................................................................ 6 4.1.1 HDFS ..................................................................................................................................................... 6 4.1.2 GFS (Google File System) ....................................................................................................................... 7 4.1.3 InterMezzo ............................................................................................................................................ 9 4.1.4 CodA .................................................................................................................................................... 10 4.1.5 Ceph.................................................................................................................................................... 12 4.1.6 DDFS .................................................................................................................................................. -
The Illusion of Space and the Science of Data Compression And
The Illusion of Space and the Science of Data Compression and Deduplication Bruce Yellin EMC Proven Professional Knowledge Sharing 2010 Bruce Yellin Advisory Technology Consultant EMC Corporation [email protected] Table of Contents What Was Old Is New Again ......................................................................................................... 4 The Business Benefits of Saving Space ....................................................................................... 6 Data Compression Strategies ....................................................................................................... 9 Data Compression Basics ....................................................................................................... 10 Compression Bakeoff .............................................................................................................. 13 Data Deduplication Strategies .................................................................................................... 16 Deduplication - Theory of Operation ....................................................................................... 16 File Level Deduplication - Single Instance Storage ................................................................. 21 Fixed-Block Deduplication ....................................................................................................... 23 Variable-Block Deduplication .................................................................................................. 24 Content-Aware Deduplication -
Designing High-Performance and Scalable Clustered Network Attached Storage with Infiniband
DESIGNING HIGH-PERFORMANCE AND SCALABLE CLUSTERED NETWORK ATTACHED STORAGE WITH INFINIBAND DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Ranjit Noronha, MS * * * * * The Ohio State University 2008 Dissertation Committee: Approved by Dhabaleswar K. Panda, Adviser Ponnuswammy Sadayappan Adviser Feng Qin Graduate Program in Computer Science and Engineering c Copyright by Ranjit Noronha 2008 ABSTRACT The Internet age has exponentially increased the volume of digital media that is being shared and distributed. Broadband Internet has made technologies such as high quality streaming video on demand possible. Large scale supercomputers also consume and cre- ate huge quantities of data. This media and data must be stored, cataloged and retrieved with high-performance. Researching high-performance storage subsystems to meet the I/O demands of applications in modern scenarios is crucial. Advances in microprocessor technology have given rise to relatively cheap off-the-shelf hardware that may be put together as personal computers as well as servers. The servers may be connected together by networking technology to create farms or clusters of work- stations (COW). The evolution of COWs has significantly reduced the cost of ownership of high-performance clusters and has allowed users to build fairly large scale machines based on commodity server hardware. As COWs have evolved, networking technologies like InfiniBand and 10 Gigabit Eth- ernet have also evolved. These networking technologies not only give lower end-to-end latencies, but also allow for better messaging throughput between the nodes. This allows us to connect the clusters with high-performance interconnects at a relatively lower cost. -
A Guide to the IBM Clustered Network File System
Front cover A Guide to the IBM Clustered Network File System Discusses the technical architecture Explains installation and configuration Includes administration and operation information Ira Chavis Dave Coutts Bob Demkowicz Jay Huie Shuzheng Liu Sheryl Qualters Daniel Turkenkopf ibm.com/redbooks Redpaper International Technical Support Organization A Guide to the IBM Clustered Network File System November 2010 REDP-4400-01 Note: Before using this information and the product it supports, read the information in “Notices” on page v. Second Edition (November 2010) This edition applies to the IBM Clustered Network File System package. © Copyright International Business Machines Corporation 2010. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . .v Trademarks . vi Preface . vii The team that wrote this paper . vii Now you can become a published author, too! . viii Comments welcome. ix Stay connected to IBM Redbooks . ix Chapter 1. Introduction to the IBM Clustered Network File System. 1 1.1 Overview . 2 1.2 IBM Clustered Network File System . 3 1.3 Business trends and line of business requirements . 4 1.3.1 Product life cycle management for the automotive industry . 4 1.3.2 Financial services . 5 1.3.3 Electronic design automation . 6 1.3.4 Communications and digital media . 6 1.3.5 Pharmaceuticals . 7 1.3.6 Other industries. 7 1.4 Challenges and customer requirements . 7 1.5 Solution elements and key features . 8 1.6 Targeted users . 9 1.7 Required skills for the reader of this paper . -
OCFS2: the Oracle Clustered File System, Version 2
OCFS2: The Oracle Clustered File System, Version 2 Mark Fasheh Oracle [email protected] Abstract Disk space is broken up into clusters which can range in size from 4 kilobytes to 1 megabyte. This talk will review the various components File data is allocated in extents of clusters. This of the OCFS2 stack, with a focus on the file allows OCFS2 a large amount of flexibility in system and its clustering aspects. OCFS2 ex- file allocation. tends many local file system features to the File metadata is allocated in blocks via a sub cluster, some of the more interesting of which allocation mechanism. All block allocators in are posix unlink semantics, data consistency, OCFS2 grow dynamically. Most notably, this shared readable mmap, etc. allows OCFS2 to grow inode allocation on de- In order to support these features, OCFS2 logi- mand. cally separates cluster access into multiple lay- ers. An overview of the low level DLM layer will be given. The higher level file system locking will be described in detail, including 1 Design Principles a walkthrough of inode locking and messaging for various operations. A small set of design principles has guided Caching and consistency strategies will be dis- most of OCFS2 development. None of them cussed. Metadata journaling is done on a per are unique to OCFS2 development, and in fact, node basis with JBD. Our reasoning behind that almost all are principles we learned from the choice will be described. Linux kernel community. They will, however, come up often in discussion of OCFS2 file sys- OCFS2 provides robust and performant recov- tem design, so it is worth covering them now. -
Sistemi Di Storage: Clustered Filesystems
Agenda ● Clustered Filesystems ● Block and Object: The Storage Evolution ● Introduzione a GlusterFS Sistemi di Storage: Clustered Filesystems Un clustered file system è un file system che può essere collegato contemporaneamente a più server. Ci sono diversi metodi per creare un cluster di server ma la maggior parte di questi non prevede l'uso del cluster file system. Quando, però, il numero di nodi cresce e la complessità del cluster aumenta, il ricorso al clustered file system come risorsa condivisa può essere la soluzione più efficace. Che cos'è un Cluster? Definizione di Cluster: Collezione di sistemi di calcolo indipendenti (workstations or PCs) collegati mediante una rete di interconnessione a basso costo (commodity interconnection Network), che viene utilizzata come una singola unificata risorsa di calcolo. Utilizzo dei Cluster Possiamo far ricadere i cluster in tre famiglie che ne caratterizzano l’utilizzo: High Availability Cluster: i nodi off rono servizi ridondanti per garantirne la disponibilita' Load Balancing Cluster: i nodi si spartiscono il carico di un determinato servizio dinamicamente High Performance Computing (HPC) Cluster: i nodi eseguono in maniera coordinata programmi paralleli che fanno un uso intenso della CPU. Load Balancing Cluster I nodi si spartiscono il carico di un determinato servizio dinamicamente High Availability Cluster I nodi off rono servizi ridondanti per garantirne la disponibilita' Tipologie di Clustered Filesystems A disco condiviso La tipologia più utilizzata di cluster file system è a disco condiviso in cui due o più server accedono contemporaneamente ad un singolo sottosistema di storage che può essere un RAID o una SAN. Sono un esempio di questa tecnologia i file system VMFS e il Global File System. -
Oracle for SAP Technology Update, Volume 22
® ® Oracle for SAP TECHNOLOGY UPDATE No. 22 Oracle for SAP, May 2013 www.oracle.com/sap Please download current version of the Oracle for SAP Technology Update Nr. 22 www.oracle.com/sap Hardware and Software CONTENTS Engineered to Work Together 3 Editorial 4 Oracle Engineered Systems for SAP 6 Oracle Exadata Database Machine for SAP customers 12 Oracle Exadata Database Machine Start Up Pack for SAP customers 14 Oracle Exalogic Elastic Cloud certified for SAP NetWeaver 7 18 SPARC SuperCluster optimized solution for SAP customers 20 ATIVAS on SPARC SuperCluster 22 Oracle Database Appliance Certified for SAP customers 24 Oracle VM server for X86 virtualization now available for SAP customers 26 Oracle Exadata for SAP at Lion 30 Oracle Exadata for SAP at Dongfeng 32 Infosys-Oracle Exadata for SAP consolidation solution 34 Atos IT – Oracle Exadata Services for SAP Customers 36 Oracle Exadata for SAP at Shenhua 38 Oracle Exadata for SAP at Koctas 42 SAP customer with Oracle Exadata Database Machine in production 45 Why SAP Customers Deploy Exadata 48 Oracle Exadata for SAP at Glencore 52 Oracle Database 11g Release 2 for SAP 58 Oracle Database Migration Technologies 71 Oracle HA Solutions Certified by SAP for Integration with SAP NetWeaver 72 Oracle Cloud File System for SAP 75 Real Application Testing for SAP customers 78 Oracle Advanced Compression at Indesit 82 Oracle Solaris Cluster 4.0 for SAP at City of Nuremberg 84 Oracle Advanced Compression at ConAgra Foods 89 Oracle RAC and ASM at EDEKA Lunar 92 Oracle Advanced Customer Services (ACS) at AkzoNobel -Triple O 94 Helsana DB2 migration to Oracle Database 96 Oracle Advanced Customer Services (ACS) for SAP 101 Oracle Advanced Customer Services (ACS) at Centrica 1021 Oracle Advanced Customer Services (ACS) O2O-NXP Semiconductors N.V. -
Best Practices for Data Sharing in a Grid Distributed SAS Environment
Best Practices for Data Sharing in a Grid Distributed SAS® Environment Updated July 2010 B E S T P R A C T I C E D OCUMENT Table of Contents 1 Abstract ................................................................................ 2 1.1 Storage performance is critical ................................................................... 2 1.2 Required background knowledge ............................................................... 2 1.3 Shared file system experience with SAS .................................................... 2 1.4 NAS Appliance (NFS or CIFS) ...................................................................... 3 1.5 Clustered and shared file system ................................................................ 5 1.6 Hybrid NAS and SAN systems - iSCSI ......................................................... 7 1.7 SAS experience with various shared file systems ..................................... 9 1.8 Comparing various storage architectures for SAS .................................. 10 2 Implementation Guidelines............................................... 11 2.1 Initial architecture design .......................................................................... 12 2.2 Throughput requirements for SAS applications ....................................... 12 2.3 Location of SASWORK and Utility Space ................................................. 12 2.4 Keep it simple ............................................................................................. 13 2.5 Design testing ............................................................................................