An Alternative Scalable Storage System
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Pete's All Things Sun (PATS): the State Of
We are in the midst of a file sys - tem revolution, and it is called ZFS. File sys- p e t e R B a e R G a Lv i n tem revolutions do not happen very often, so when they do, excitement ensues— Pete’s all things maybe not as much excitement as during a political revolution, but file system revolu- Sun (PATS): the tions are certainly exciting for geeks. What are the signs that we are in a revolution? By state of ZFS my definition, a revolution starts when the Peter Baer Galvin (www.galvin.info) is the Chief peasants (we sysadmins) are unhappy with Technologist for Corporate Technologies, a premier the status quo, some group comes up with systems integrator and VAR (www.cptech.com). Be- fore that, Peter was the systems manager for Brown a better idea, and the idea spreads beyond University’s Computer Science Department. He has written articles and columns for many publications that group and takes on a life of its own. Of and is coauthor of the Operating Systems Concepts course, in a successful revolution the new and Applied Operating Systems Concepts textbooks. As a consultant and trainer, Peter teaches tutorials idea actually takes hold and does improve and gives talks on security and system administra- tion worldwide. the peasant’s lot. [email protected] ;login: has had two previous articles about ZFS. The first, by Tom Haynes, provided an overview of ZFS in the context of building a home file server (;login:, vol. 31, no. 3). In the second, Dawidek and McKusick (;login:, vol. -
Storage Administration Guide Storage Administration Guide SUSE Linux Enterprise Server 12 SP4
SUSE Linux Enterprise Server 12 SP4 Storage Administration Guide Storage Administration Guide SUSE Linux Enterprise Server 12 SP4 Provides information about how to manage storage devices on a SUSE Linux Enterprise Server. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii 1 Available Documentation xii 2 Giving Feedback xiv 3 Documentation Conventions xiv 4 Product Life Cycle and Support xvi Support Statement for SUSE Linux Enterprise Server xvii • Technology Previews xviii I FILE SYSTEMS AND MOUNTING 1 1 Overview -
The Title Title: Subtitle March 2007
sub title The Title Title: Subtitle March 2007 Copyright c 2006-2007 BSD Certification Group, Inc. Permission to use, copy, modify, and distribute this documentation for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE DOCUMENTATION IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS DOCUMENTATION INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CON- SEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEG- LIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS DOCUMENTATION. NetBSD and pkgsrc are registered trademarks of the NetBSD Foundation, Inc. FreeBSD is a registered trademark of the FreeBSD Foundation. Contents Introduction vii 1 Installing and Upgrading the OS and Software 1 1.1 Recognize the installation program used by each operating system . 2 1.2 Recognize which commands are available for upgrading the operating system 6 1.3 Understand the difference between a pre-compiled binary and compiling from source . 8 1.4 Understand when it is preferable to install a pre-compiled binary and how to doso ...................................... 9 1.5 Recognize the available methods for compiling a customized binary . 10 1.6 Determine what software is installed on a system . 11 1.7 Determine which software requires upgrading . 12 1.8 Upgrade installed software . 12 1.9 Determine which software have outstanding security advisories . -
NVIDIA Magnum IO Gpudirect Storage
NVIDIA Magnum IO GPUDirect Storage Installation and Troubleshooting Guide TB-10112-001_v1.0.0 | August 2021 Table of Contents Chapter 1. Introduction........................................................................................................ 1 Chapter 2. Installing GPUDirect Storage.............................................................................2 2.1. Before You Install GDS.............................................................................................................2 2.2. Installing GDS............................................................................................................................3 2.2.1. Removal of Prior GDS Installation on Ubuntu Systems...................................................3 2.2.2. Preparing the OS................................................................................................................3 2.2.3. GDS Package Installation.................................................................................................. 4 2.2.4. Verifying the Package Installation.....................................................................................4 2.2.5. Verifying a Successful GDS Installation............................................................................5 2.3. Installed GDS Libraries and Tools...........................................................................................6 2.4. Uninstalling GPUDirect Storage...............................................................................................7 2.5. Environment -
How Netflix Tunes EC2 Instances for Performance
CMP325 How Netflix Tunes EC2 Instances for Performance Brendan Gregg, Performance and OS Engineering Team November 28, 2017 © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Netflix performance and operating systems team • Evaluate technology - Instance types, Amazon Elastic Compute Cloud (EC2) options • Recommendations and best practices - Instance kernel tuning, assist app tuning • Develop performance tools - Develop tools for observability and analysis • Project support - New database, programming language, software change • Incident response - Performance issues, scalability issues © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda 1. Instance selection 2. Amazon EC2 features 3. Kernel tuning 4. Methodologies 5. Observability © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Warnings • This is what’s in our medicine cabinet • Consider these “best before: 2018” • Take only if prescribed by a performance engineer © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1. Instance selection © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Netflix cloud Many application workloads: Compute, storage, caching… EC2 Applications (services) S3 ELB Elasticsearch Cassandra EVCache SES SQS © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Netflix AWS environment • Elastic Load Balancing ASG Cluster allows real load testing prod1 ELB 1. Single instance canary, then, Canary 2. Auto scaling group • Much better than micro- ASG-v010 ASG-v011 benchmarking alone, which … … is error prone Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. -
Setup Software RAID1 Array on Running Centos 6.3 Using Mdadm
Setup software RAID1 array on running CentOS 6.3 using mdadm. (Multiple Device Administrator) All commands run from terminal as super user. Default CentOS 6.3 installation with two hard drives, /dev/sda and /dev/sdb which are identical in size. Machine name is “serverbox.local”. /dev/sdb is currently unused, and /dev/sda has the following partitions: /dev/sda1: /boot partition, ext4; /dev/sda2: is used for LVM (volume group vg_serverbox) and contains / (volume root), swap (volume swap_1) and /home (volume home). Final RAID1 configuration: /dev/md0 (made up of /dev/sda1 and /dev/sdb1): /boot partition, ext4; /dev/md1 (made up of /dev/sda2 and /dev/sdb2): LVM (volume group vg_serverbox), contains / (volume root), swap (volume swap_1) and /home (volume home). 1. Gather information about current system. Report the current disk space usage: df -h View physical disks: fdisk -l View physical volumes on logical disk partition: pvdisplay View virtual group details: vgdisplay View Logical volumes: lvdisplay Load kernel modules (to avoid a reboot): modprobe linear modprobe raid0 modprobe raid1 Verify personalities: cat /proc/mdstat The output should look as follows: serverbox:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] unused devices: <none> 2. Preparing /dev/sdb To create a RAID1 array on a running system, prepare the /dev/sdb hard drive for RAID1, then copy the contents of /dev/sda hard drive to it, and finally add /dev/sda to the RAID1 array. Copy the partition table from /dev/sda -
Setting up Software RAID in Ubuntu Server | Secu
Setting up software RAID in Ubuntu Server | Secu... http://advosys.ca/viewpoints/2007/04/setting-up-so... Setting up software RAID in Ubuntu Server April 24th, 2007 Posted by Derrick Webber Updated Mar 13 2009 to reflect improvements in Ubuntu 8.04 and later. Linux has excellent software-based RAID built into the kernel. Unfortunately information on configuring and maintaining it is sparse. Back in 2003, O’Reilly published Managing RAID on Linux and that book is still mostly up to date, but finding clear instructions on the web for setting up RAID has become a chore. Here is how to install Ubuntu Server with software RAID 1 (disk mirroring). This guide has been tested on Ubuntu Server 8.04 LTS (Hardy Heron). I strongly recommend using Ubuntu Hardy or later if you want to boot from RAID1. Software RAID vs. hardware RAID Some system administrators still sneer at the idea of software RAID. Years ago CPUs didn’t have the speed to manage both a busy server and RAID activities. That’s not true any more, especially when all you want to do is mirror a drive with RAID1. Linux software RAID is ideal for mirroring, and due to kernel disk caching and buffering it can actually be faster than RAID1 on lower end RAID hardware. However, for larger requirements like RAID 5, the CPU can still get bogged down with software RAID. Software RAID is inexpensive to implement: no need for expensive controllers or identical drives. Software RAID works 1 de 23 27/09/09 13:41 Setting up software RAID in Ubuntu Server | Secu.. -
Ubuntu Server Guide Basic Installation Preparing to Install
Ubuntu Server Guide Welcome to the Ubuntu Server Guide! This site includes information on using Ubuntu Server for the latest LTS release, Ubuntu 20.04 LTS (Focal Fossa). For an offline version as well as versions for previous releases see below. Improving the Documentation If you find any errors or have suggestions for improvements to pages, please use the link at thebottomof each topic titled: “Help improve this document in the forum.” This link will take you to the Server Discourse forum for the specific page you are viewing. There you can share your comments or let us know aboutbugs with any page. PDFs and Previous Releases Below are links to the previous Ubuntu Server release server guides as well as an offline copy of the current version of this site: Ubuntu 20.04 LTS (Focal Fossa): PDF Ubuntu 18.04 LTS (Bionic Beaver): Web and PDF Ubuntu 16.04 LTS (Xenial Xerus): Web and PDF Support There are a couple of different ways that the Ubuntu Server edition is supported: commercial support and community support. The main commercial support (and development funding) is available from Canonical, Ltd. They supply reasonably- priced support contracts on a per desktop or per-server basis. For more information see the Ubuntu Advantage page. Community support is also provided by dedicated individuals and companies that wish to make Ubuntu the best distribution possible. Support is provided through multiple mailing lists, IRC channels, forums, blogs, wikis, etc. The large amount of information available can be overwhelming, but a good search engine query can usually provide an answer to your questions. -
Performance and Scalability of a Sensor Data Storage Framework
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Aaltodoc Publication Archive Aalto University School of Science Degree Programme in Computer Science and Engineering Paul Tötterman Performance and Scalability of a Sensor Data Storage Framework Master’s Thesis Helsinki, March 10, 2015 Supervisor: Associate Professor Keijo Heljanko Advisor: Lasse Rasinen M.Sc. (Tech.) Aalto University School of Science ABSTRACT OF Degree Programme in Computer Science and Engineering MASTER’S THESIS Author: Paul Tötterman Title: Performance and Scalability of a Sensor Data Storage Framework Date: March 10, 2015 Pages: 67 Major: Software Technology Code: T-110 Supervisor: Associate Professor Keijo Heljanko Advisor: Lasse Rasinen M.Sc. (Tech.) Modern artificial intelligence and machine learning applications build on analysis and training using large datasets. New research and development does not always start with existing big datasets, but accumulate data over time. The same storage solution does not necessarily cover the scale during the lifetime of the research, especially if scaling up from using common workgroup storage technologies. The storage infrastructure at ZenRobotics has grown using standard workgroup technologies. The current approach is starting to show its limits, while the storage growth is predicted to continue and accelerate. Successful capacity planning and expansion requires a better understanding of the patterns of the use of storage and its growth. We have examined the current storage architecture and stored data from different perspectives in order to gain a better understanding of the situation. By performing a number of experiments we determine key properties of the employed technologies. The combination of these factors allows us to make informed decisions about future storage solutions. -
OS7 User's Manual
OS7 User’s Manual N2810 Series/ N4810 Series/N5810 Series/ N4910U/N4910U PRO/N4820U N12850 Series/N16850 Series/ N8910/N12910/N12910SAS/N16910SAS Copyright and Trademark Notice Thecus and other names of Thecus products are registered trademarks of Thecus Technology Corp. Microsoft, Windows, and the Windows logo are registered trademarks of Microsoft Corporation. Apple, iTunes and Apple OS X are registered trademarks of Apple Computers, Inc. All other trademarks and brand names are the property of their respective owners. Specifications are subject to change without notice. Copyright © 2017 Thecus Technology Corporation. All rights reserved. About This Manual All information in this manual has been carefully verified to ensure its correctness. In case of an error, please provide us with your feedback. Thecus Technology Corporation reserves the right to modify the contents of this manual without notice. Product name: Thecus OS7.0 used models Manual Version: 1.5.1 Release Date: 2017/06 Limited Warranty Thecus Technology Corporation guarantees all components of Thecus NAS products are thoroughly tested before they leave the factory and should function normally under general usage. In case of any system malfunctions, Thecus Technology Corporation and its local representatives and dealers are responsible for repair without cost to the customer if the product fails within the warranty period and under normal usage. Thecus Technology Corporation is not responsible for any damage or loss of data deemed to be caused by its products. It is highly recommended that users conduct necessary back-up practices. Check the functions that are available on your particular Thecus NAS model at: http://www.Thecus.com 2 Safety Warnings For your safety, please read and follow the following safety warnings: ● Read this manual thoroughly before attempting to set up your Thecus IP storage. -
Software-RAID-HOWTO.Pdf
Software-RAID-HOWTO Software-RAID-HOWTO Table of Contents The Software-RAID HOWTO...........................................................................................................................1 Jakob Østergaard [email protected] and Emilio Bueso [email protected] 1. Introduction..........................................................................................................................................1 2. Why RAID?.........................................................................................................................................1 3. Devices.................................................................................................................................................1 4. Hardware issues...................................................................................................................................1 5. RAID setup..........................................................................................................................................1 6. Detecting, querying and testing...........................................................................................................2 7. Tweaking, tuning and troubleshooting................................................................................................2 8. Reconstruction.....................................................................................................................................2 9. Performance.........................................................................................................................................2 -
Quick Reference 11 Unix Linux Troubleshooting
Quick Reference Card 11 Linux Troubleshooting Your? host logo Boot Process #Boot steps: Recover from an unknown root password BIOS #EFI is successor for e.g. IA64, #1) Start single user mode, then passwd #Extensible Firmware Interface #2) Boot from a Linux CD, su -, mount / partition: mount /dev/hda3 /mnt mbr #Create backup (if is input file, of=output) # Remove the x in the /mnt/etc/passwd file (second field on the root line) dd if=/dev/hda of=/tmp/mbr bs=512 count=1 #3) Run passwd in the Rescue Mode file /tmp/mbr #Check -> x86 boot sector #4) Add kernel boot parameter: init=/bin/sh hexdump /tmp/mbr #See inside copied mbr #See also: Lab ad endum at Roberts_Quick_References khexdump /tmp/mbr #Install via: yast2 -i kdeutils-extra, or ghex2 /tmp/mbr #Package: rpm -qf $(which ghex2) Fix bootloader #Last 2 Bytes 'magic number': 55 AA #Start in Rescue Mode, and at the first prompt: boot loader #GRUB or LILO. #Type zast or yast #German keyboard work around :-) #GRUB supports TFTP network boot, serial console, shell: #System, Boot Loader Configuration, Reset <Alt-e>, #In bootgui: press <Esc>, <C> grub command line, or grub from Linux CLI #Propose New Configuration, Finish <Alt-f> help #Show GRUB commands find /boot/vmlinuz #Returns partition with kernel YaST boot into system #To live with a damaged boot loader find /etc/fstab #Returns / partition (Starts with 0, not 1) #Start from (any version) CD1 #Tip: Hardcode IDE disks in BIOS, not on 'automatic' #Start Installation, License Agreement, Language, Boot Installed System #Kernel options: less /usr/src/linux/Documentation/kernel-parameters.txt Rescue Mode #Change setup of a non bootable machine: vi /boot/grub/device.map #Map GRUB names to Linux names, e.g.: #Boot from CD1 (any version, highest SP for driver support) (hd0) /dev/hda #Select Rescue System, and login as root grub-md5-crypt #Create encrypted password for menu.lst grub #Find the / partition: init #PID 1 find /etc/fstab exit Software RAID #Mirror of /boot: LILO only, not GRUB! mount /dev/hda3 /mnt #Mount / partition #Hardware RAID is prefer., if not SATA, e.g.