CIFS SMB2 SMB3 Meets Linux a Year in Review
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Elastic Storage for Linux on System Z IBM Research & Development Germany
Peter Münch T/L Test and Development – Elastic Storage for Linux on System z IBM Research & Development Germany Elastic Storage for Linux on IBM System z © Copyright IBM Corporation 2014 9.0 Elastic Storage for Linux on System z Session objectives • This presentation introduces the Elastic Storage, based on General Parallel File System technology that will be available for Linux on IBM System z. Understand the concepts of Elastic Storage and which functions will be available for Linux on System z. Learn how you can integrate and benefit from the Elastic Storage in a Linux on System z environment. Finally, get your first impression in a live demo of Elastic Storage. 2 © Copyright IBM Corporation 2014 Elastic Storage for Linux on System z Trademarks The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. AIX* FlashSystem Storwize* Tivoli* DB2* IBM* System p* WebSphere* DS8000* IBM (logo)* System x* XIV* ECKD MQSeries* System z* z/VM* * Registered trademarks of IBM Corporation The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. -
BESIII Physical Analysis on Hadoop Platform
20th International Conference on Computing in High Energy and Nuclear Physics (CHEP2013) IOP Publishing Journal of Physics: Conference Series 513 (2014) 032044 doi:10.1088/1742-6596/513/3/032044 BESIII Physical Analysis on Hadoop Platform Jing HUO12, Dongsong ZANG12, Xiaofeng LEI12, Qiang LI12, Gongxing SUN1 1Institute of High Energy Physics, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China E-mail: [email protected] Abstract. In the past 20 years, computing cluster has been widely used for High Energy Physics data processing. The jobs running on the traditional cluster with a Data-to-Computing structure, have to read large volumes of data via the network to the computing nodes for analysis, thereby making the I/O latency become a bottleneck of the whole system. The new distributed computing technology based on the MapReduce programming model has many advantages, such as high concurrency, high scalability and high fault tolerance, and it can benefit us in dealing with Big Data. This paper brings the idea of using MapReduce model to do BESIII physical analysis, and presents a new data analysis system structure based on Hadoop platform, which not only greatly improve the efficiency of data analysis, but also reduces the cost of system building. Moreover, this paper establishes an event pre-selection system based on the event level metadata(TAGs) database to optimize the data analyzing procedure. 1. Introduction 1.1 The current BESIII computing architecture High Energy Physics experiment is a typical data-intensive application. The BESIII computing system now consist of 3PB+ data, 6500 CPU cores, and it is estimated that there will be more than 10PB data produced in the future 5 years. -
Membrane: Operating System Support for Restartable File Systems Swaminathan Sundararaman, Sriram Subramanian, Abhishek Rajimwale, Andrea C
Membrane: Operating System Support for Restartable File Systems Swaminathan Sundararaman, Sriram Subramanian, Abhishek Rajimwale, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, Michael M. Swift Computer Sciences Department, University of Wisconsin, Madison Abstract and most complex code bases in the kernel. Further, We introduce Membrane, a set of changes to the oper- file systems are still under active development, and new ating system to support restartable file systems. Mem- ones are introduced quite frequently. For example, Linux brane allows an operating system to tolerate a broad has many established file systems, including ext2 [34], class of file system failures and does so while remain- ext3 [35], reiserfs [27], and still there is great interest in ing transparent to running applications; upon failure, the next-generation file systems such as Linux ext4 and btrfs. file system restarts, its state is restored, and pending ap- Thus, file systems are large, complex, and under develop- plication requests are serviced as if no failure had oc- ment, the perfect storm for numerous bugs to arise. curred. Membrane provides transparent recovery through Because of the likely presence of flaws in their imple- a lightweight logging and checkpoint infrastructure, and mentation, it is critical to consider how to recover from includes novel techniques to improve performance and file system crashes as well. Unfortunately, we cannot di- correctness of its fault-anticipation and recovery machin- rectly apply previous work from the device-driver litera- ery. We tested Membrane with ext2, ext3, and VFAT. ture to improving file-system fault recovery. File systems, Through experimentation, we show that Membrane in- unlike device drivers, are extremely stateful, as they man- duces little performance overhead and can tolerate a wide age vast amounts of both in-memory and persistent data; range of file system crashes. -
Fortran Reference Guide
FORTRAN REFERENCE GUIDE Version 2018 TABLE OF CONTENTS Preface............................................................................................................ xv Audience Description......................................................................................... xv Compatibility and Conformance to Standards............................................................ xv Organization................................................................................................... xvi Hardware and Software Constraints...................................................................... xvii Conventions................................................................................................... xvii Related Publications........................................................................................ xviii Chapter 1. Language Overview............................................................................... 1 1.1. Elements of a Fortran Program Unit.................................................................. 1 1.1.1. Fortran Statements................................................................................. 1 1.1.2. Free and Fixed Source............................................................................. 2 1.1.3. Statement Ordering................................................................................. 2 1.2. The Fortran Character Set.............................................................................. 3 1.3. Free Form Formatting.................................................................................. -
Introduction to Linux on System Z
IBM Linux and Technology Center Introduction to Linux on System z Mario Held IBM Lab Boeblingen, Germany © 2009 IBM Corporation IBM Linux and Technology Center Trademarks The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both. Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market. Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: *, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter® The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. -
Developments in GFS2
Developments in GFS2 Andy Price <[email protected]> Software Engineer, GFS2 OSSEU 2018 11 GFS2 recap ● Shared storage cluster filesystem ● High availability clusters ● Uses glocks (“gee-locks”) based on DLM locking ● One journal per node ● Divided into resource groups ● Requires clustered lvm: clvmd/lvmlockd ● Userspace tools: gfs2-utils 22 Not-so-recent developments ● Full SELinux support (linux 4.5) ● Previously no way to tell other nodes that cached labels are invalid ● Workaround was to mount with -o context=<label> ● Relabeling now causes security label invalidation across cluster 33 Not-so-recent developments ● Resource group stripe alignment (gfs2-utils 3.1.11) ● mkfs.gfs2 tries to align resource groups to RAID stripes where possible ● Uses libblkid topology information to find stripe unit and stripe width values ● Tries to spread resource groups evenly across disks 44 Not-so-recent developments ● Resource group LVBs (linux 3.6) ● -o rgrplvb (all nodes) ● LVBs: data attached to DLM lock requests ● Allows resource group metadata to be cached and transferred with glocks ● Decreases resource group-related I/O 55 Not-so-recent developments ● Location-based readdir cookies (linux 4.5) ● -o loccookie (all nodes) ● Uniqueness required by NFS ● Previously 31 bits of filename hash ● Problem: collisions with large directories (100K+ files) ● Cookies now based on the location of the directory entry within the directory 66 Not-so-recent developments ● gfs_controld is gone (linux 3.3) ● dlm now triggers gfs2 journal recovery via callbacks ● -
Globalfs: a Strongly Consistent Multi-Site File System
GlobalFS: A Strongly Consistent Multi-Site File System Leandro Pacheco Raluca Halalai Valerio Schiavoni University of Lugano University of Neuchatelˆ University of Neuchatelˆ Fernando Pedone Etienne Riviere` Pascal Felber University of Lugano University of Neuchatelˆ University of Neuchatelˆ Abstract consistency, availability, and tolerance to partitions. Our goal is to ensure strongly consistent file system operations This paper introduces GlobalFS, a POSIX-compliant despite node failures, at the price of possibly reduced geographically distributed file system. GlobalFS builds availability in the event of a network partition. Weak on two fundamental building blocks, an atomic multicast consistency is suitable for domain-specific applications group communication abstraction and multiple instances of where programmers can anticipate and provide resolution a single-site data store. We define four execution modes and methods for conflicts, or work with last-writer-wins show how all file system operations can be implemented resolution methods. Our rationale is that for general-purpose with these modes while ensuring strong consistency and services such as a file system, strong consistency is more tolerating failures. We describe the GlobalFS prototype in appropriate as it is both more intuitive for the users and detail and report on an extensive performance assessment. does not require human intervention in case of conflicts. We have deployed GlobalFS across all EC2 regions and Strong consistency requires ordering commands across show that the system scales geographically, providing replicas, which needs coordination among nodes at performance comparable to other state-of-the-art distributed geographically distributed sites (i.e., regions). Designing file systems for local commands and allowing for strongly strongly consistent distributed systems that provide good consistent operations over the whole system. -
Add-Ons for Red Hat Enterprise Linux
DATASHEET ADD-ONS FOR RED HAT ENTERPRISE LINUX WORKLOAD AND MISSION-CRITICAL PLATFORM ENHANCEMENTS In conjunction with Red Hat® Enterprise Linux®, Red Hat offers a portfolio of Add-Ons to extend the features of your Red Hat Enterprise Linux subscription. Add-Ons to Red Hat Enterprise Linux tailor your application environment to suit your particular computing requirements. With increased flexibility and choice, customers can deploy what they need, when they need it. ADD-ON OPTIONS FOR AVAILABILITY MANAGEMENT High Availability Add-On Red Hat’s High Availability Add-On provides on-demand failover services between nodes within a cluster, making applications highly available. The High Availability Add-On supports up to 16 nodes and may be configured for most applications that use customizable agents, as well as for virtual guests. The High Availability Add-On also includes failover support for off-the-shelf applications like Apache, MySQL, and PostgreSQL. When using the High Availability Add-On, a highly available service can fail over from one node to another with no apparent interruption to cluster clients. The High Availability Add-On also ensures data integrity when one cluster node takes over control of a service from another clus- ter node. It achieves this by promptly evicting nodes from the cluster that are deemed to be faulty using a method called “fencing” that prevents data corruption. www.redhat.com DATASHEET ADD-ONS For RED Hat ENterprise LINUX 6 Resilient Storage Add-On Red Hat’s Resilient Storage Add-On enables a shared storage or clustered file system to access the same storage device over a network. -
AIX Migration to Cloud with IBM Power Virtual Server
AIX Migration to Cloud with IBM Power Virtual Server An IBM Systems Lab Services Tutorial Aaron Bolding Berjis Patel Vess Natchev [email protected] TABLE OF CONTENTS CHAPTER 1: SOLUTION OVERVIEW............................. 1 Introduction ................................................................................ 1 Use Cases .................................................................................. 1 Migration via PowerVC OVA ..................................................... 1 Transfer System Backup Using the Public Internet ..................... 2 Transfer System Backup Using Cloud Object Storage ................. 2 Solution Components and Requirements ........................................ 2 Components .......................................................................... 2 Migration via PowerVC OVA ..................................................... 2 Transfer System Backup Using the Public Internet ..................... 2 Transfer System Backup Using Cloud Object Storage ................. 2 Requirements ........................................................................ 3 Solution Diagrams ....................................................................... 3 Transfer System Backup Using the Public Internet ..................... 3 Transfer System Backup Using Cloud Object Storage ................. 4 CHAPTER 2: IMPLEMENTATION .................................. 5 Migration via PowerVC OVA .......................................................... 5 Procedure to Configure IBM Cloud Object Storage ..................... -
Veritas Access Administrator's Guide
Veritas Access Administrator's Guide 7.4.2 Linux To be used with the Veritas Access Appliance v7.4.2 (3340) documentation April 2021 Veritas Access Administrator's Guide Last updated: 2021-04-15 Document version: 7.4.2 Rev 5 Legal Notice Copyright © 2018 Veritas Technologies LLC. All rights reserved. Veritas, the Veritas Logo, Veritas InfoScale, and NetBackup are trademarks or registered trademarks of Veritas Technologies LLC or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This product may contain third-party software for which Veritas is required to provide attribution to the third party (“Third-Party Programs”). Some of the Third-Party Programs are available under open source or free software licenses. The License Agreement accompanying the Software does not alter any rights or obligations you may have under those open source or free software licenses. Refer to the third-party legal notices document accompanying this Veritas product or available at: https://www.veritas.com/licensing/process The product described in this document is distributed under licenses restricting its use, copying, distribution, and decompilation/reverse engineering. No part of this document may be reproduced in any form by any means without prior written authorization of Veritas Technologies LLC and its licensors, if any. THE DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. VERITAS TECHNOLOGIES LLC SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. -
RACF Command Tips
RACF Command Tips SHARE ‐ March 2015 Session 18875 RSH Consulting ‐ Robert S. Hansel RSH Consulting, Inc. is an IT security professional services firm established in 1992 and dedicated to helping clients strengthen their IBM z/OS mainframe access controls by fully exploiting all the capabilities and latest innovations in RACF. RSH's services include RACF security reviews and audits, initial implementation of new controls, enhancement and remediation of existing controls, and training. • www.rshconsulting.com • 617‐969‐9050 Robert S. Hansel is Lead RACF Specialist and founder of RSH Consulting, Inc. He began working with RACF in 1986 and has been a RACF administrator, manager, auditor, instructor, developer, and consultant. Mr. Hansel is especially skilled at redesigning and refining large‐scale implementations of RACF using role‐based access control concepts. He is a leading expert in securing z/OS Unix using RACF. Mr. Hansel has created elaborate automated tools to assist clients with RACF administration, database merging, identity management, and quality assurance. • 617‐969‐8211 • [email protected] • www.linkedin.com/in/roberthansel • http://twitter.com/RSH_RACF RACF Command Tips SHARE 2 © 2016 RSH Consulting, Inc. All Rights Reserved. March 2016 Topics . User Commands . Group Commands . Dataset Command . General Resource Commands . PERMIT Command . Generic Profile Refresh . List Commands . SEARCH Command . Console Command Entry . Building Commands with Microsoft Excel RACF and z/OS are Trademarks of the International Business Machines Corporation RACF Command Tips SHARE 3 © 2016 RSH Consulting, Inc. All Rights Reserved. March 2016 User Commands . ADDUSER Defaults: • OWNER ‐ Creator's ID • DFLTGRP ‐ Creator's Current Connect Group • PASSWORD ‐ Pre‐z/OS 2.2: Default Group z/OS 2.2: NOPASSWORD • Always specify when creating new ID . -
Implementing Nfsv4 in the Enterprise: Planning and Migration Strategies
Front cover Implementing NFSv4 in the Enterprise: Planning and Migration Strategies Planning and implementation examples for AFS and DFS migrations NFSv3 to NFSv4 migration examples NFSv4 updates in AIX 5L Version 5.3 with 5300-03 Recommended Maintenance Package Gene Curylo Richard Joltes Trishali Nayar Bob Oesterlin Aniket Patel ibm.com/redbooks International Technical Support Organization Implementing NFSv4 in the Enterprise: Planning and Migration Strategies December 2005 SG24-6657-00 Note: Before using this information and the product it supports, read the information in “Notices” on page xi. First Edition (December 2005) This edition applies to Version 5, Release 3, of IBM AIX 5L (product number 5765-G03). © Copyright International Business Machines Corporation 2005. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . xi Trademarks . xii Preface . xiii The team that wrote this redbook. xiv Acknowledgments . xv Become a published author . xvi Comments welcome. xvii Part 1. Introduction . 1 Chapter 1. Introduction. 3 1.1 Overview of enterprise file systems. 4 1.2 The migration landscape today . 5 1.3 Strategic and business context . 6 1.4 Why NFSv4? . 7 1.5 The rest of this book . 8 Chapter 2. Shared file system concepts and history. 11 2.1 Characteristics of enterprise file systems . 12 2.1.1 Replication . 12 2.1.2 Migration . 12 2.1.3 Federated namespace . 13 2.1.4 Caching . 13 2.2 Enterprise file system technologies. 13 2.2.1 Sun Network File System (NFS) . 13 2.2.2 Andrew File System (AFS) .