Host Sustems SESUG
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Fragmentation in Large Object Repositories
Fragmentation in Large Object Repositories Experience Paper Russell Sears Catharine van Ingen University of California, Berkeley Microsoft Research [email protected] [email protected] ABSTRACT 1. INTRODUCTION Fragmentation leads to unpredictable and degraded application Application data objects continue to increase in size. performance. While these problems have been studied in detail Furthermore, the increasing popularity of web services and other for desktop filesystem workloads, this study examines newer network applications means that systems that once managed static systems such as scalable object stores and multimedia archives of “finished” objects now manage frequently modified repositories. Such systems use a get/put interface to store objects. versions of application data. Rather than updating these objects in In principle, databases and filesystems can support such place, typical archives either store multiple versions of the objects applications efficiently, allowing system designers to focus on (the V of WebDAV stands for “versioning” [25]), or simply do complexity, deployment cost and manageability. wholesale replacement (as in SharePoint Team Services [19]). Similarly, applications such as personal video recorders and Although theoretical work proves that certain storage policies media subscription servers continuously allocate and delete large, behave optimally for some workloads, these policies often behave transient objects. poorly in practice. Most storage benchmarks focus on short-term behavior or do not measure fragmentation. We compare SQL Applications store large objects as some combination of files in Server to NTFS and find that fragmentation dominates the filesystem and as BLOBs (binary large objects) in a database. performance when object sizes exceed 256KB-1MB. NTFS Only folklore is available regarding the tradeoffs. -
ECE 598 – Advanced Operating Systems Lecture 19
ECE 598 { Advanced Operating Systems Lecture 19 Vince Weaver http://web.eece.maine.edu/~vweaver [email protected] 7 April 2016 Announcements • Homework #7 was due • Homework #8 will be posted 1 Why use FAT over ext2? • FAT simpler, easy to code • FAT supported on all major OSes • ext2 faster, more robust filename and permissions 2 btrfs • B-tree fs (similar to a binary tree, but with pages full of leaves) • overwrite filesystem (overwite on modify) vs CoW • Copy on write. When write to a file, old data not overwritten. Since old data not over-written, crash recovery better Eventually old data garbage collected • Data in extents 3 • Copy-on-write • Forest of trees: { sub-volumes { extent-allocation { checksum tree { chunk device { reloc • On-line defragmentation • On-line volume growth 4 • Built-in RAID • Transparent compression • Snapshots • Checksums on data and meta-data • De-duplication • Cloning { can make an exact snapshot of file, copy-on- write different than link, different inodles but same blocks 5 Embedded • Designed to be small, simple, read-only? • romfs { 32 byte header (magic, size, checksum,name) { Repeating files (pointer to next [0 if none]), info, size, checksum, file name, file data • cramfs 6 ZFS Advanced OS from Sun/Oracle. Similar in idea to btrfs indirect still, not extent based? 7 ReFS Resilient FS, Microsoft's answer to brtfs and zfs 8 Networked File Systems • Allow a centralized file server to export a filesystem to multiple clients. • Provide file level access, not just raw blocks (NBD) • Clustered filesystems also exist, where multiple servers work in conjunction. -
Ext4 File System and Crash Consistency
1 Ext4 file system and crash consistency Changwoo Min 2 Summary of last lectures • Tools: building, exploring, and debugging Linux kernel • Core kernel infrastructure • Process management & scheduling • Interrupt & interrupt handler • Kernel synchronization • Memory management • Virtual file system • Page cache and page fault 3 Today: ext4 file system and crash consistency • File system in Linux kernel • Design considerations of a file system • History of file system • On-disk structure of Ext4 • File operations • Crash consistency 4 File system in Linux kernel User space application (ex: cp) User-space Syscalls: open, read, write, etc. Kernel-space VFS: Virtual File System Filesystems ext4 FAT32 JFFS2 Block layer Hardware Embedded Hard disk USB drive flash 5 What is a file system fundamentally? int main(int argc, char *argv[]) { int fd; char buffer[4096]; struct stat_buf; DIR *dir; struct dirent *entry; /* 1. Path name -> inode mapping */ fd = open("/home/lkp/hello.c" , O_RDONLY); /* 2. File offset -> disk block address mapping */ pread(fd, buffer, sizeof(buffer), 0); /* 3. File meta data operation */ fstat(fd, &stat_buf); printf("file size = %d\n", stat_buf.st_size); /* 4. Directory operation */ dir = opendir("/home"); entry = readdir(dir); printf("dir = %s\n", entry->d_name); return 0; } 6 Why do we care EXT4 file system? • Most widely-deployed file system • Default file system of major Linux distributions • File system used in Google data center • Default file system of Android kernel • Follows the traditional file system design 7 History of file system design 8 UFS (Unix File System) • The original UNIX file system • Design by Dennis Ritche and Ken Thompson (1974) • The first Linux file system (ext) and Minix FS has a similar layout 9 UFS (Unix File System) • Performance problem of UFS (and the first Linux file system) • Especially, long seek time between an inode and data block 10 FFS (Fast File System) • The file system of BSD UNIX • Designed by Marshall Kirk McKusick, et al. -
Openvms: an Introduction
The Operating System Handbook or, Fake Your Way Through Minis and Mainframes by Bob DuCharme VMS Table of Contents Chapter 7 OpenVMS: An Introduction.............................................................................. 7.1 History..........................................................................................................................2 7.1.1 Today........................................................................................................................3 7.1.1.1 Popular VMS Software..........................................................................................4 7.1.2 VMS, DCL................................................................................................................4 Chapter 8 Getting Started with OpenVMS........................................................................ 8.1 Starting Up...................................................................................................................7 8.1.1 Finishing Your VMS Session...................................................................................7 8.1.1.1 Reconnecting..........................................................................................................7 8.1.2 Entering Commands..................................................................................................8 8.1.2.1 Retrieving Previous Commands............................................................................9 8.1.2.2 Aborting Screen Output.........................................................................................9 -
MS-DOS Basics.Pdf
MS-DOS Basics The Command Prompt When you first turn on your computer, you will see some cryptic information flash by. MS-DOS displays this information to let you know how it is configuring your computer. You can ignore it for now. When the information stops scrolling past, you'll see the following: C:\> This is called the command prompt or DOS prompt. The flashing underscore next to the command prompt is called the cursor. The cursor shows where the command you type will appear. Type the following command at the command prompt: ver The following message appears on your screen: MS-DOS version 6.22 Viewing the Contents of a Directory To view the contents of a directory 1. Type the following at the command prompt: dir A list similar to the following appears: Changing Directories To change from the root directory to the WINDOWS directory To change directories, you will use the cd command. The cd command stands for "change directory." 1. Type the following at the command prompt: cd windows The command prompt changes. It should now look like the following: C:\WINDOWS> Next, you will use the dir command to view a list of the files in the DOS directory. Viewing the Contents of WINDOWS Directory To view a list of the files in the WINDOWS directory 1. Type the following at the command prompt: dir Changing Back to the Root Directory To change to the root directory 1. Type the following at the command prompt: cd \ Note that the slash you type in this command is a backslash (\), not a forward slash (/). -
Filesystem Considerations for Embedded Devices ELC2015 03/25/15
Filesystem considerations for embedded devices ELC2015 03/25/15 Tristan Lelong Senior embedded software engineer Filesystem considerations ABSTRACT The goal of this presentation is to answer a question asked by several customers: which filesystem should you use within your embedded design’s eMMC/SDCard? These storage devices use a standard block interface, compatible with traditional filesystems, but constraints are not those of desktop PC environments. EXT2/3/4, BTRFS, F2FS are the first of many solutions which come to mind, but how do they all compare? Typical queries include performance, longevity, tools availability, support, and power loss robustness. This presentation will not dive into implementation details but will instead summarize provided answers with the help of various figures and meaningful test results. 2 TABLE OF CONTENTS 1. Introduction 2. Block devices 3. Available filesystems 4. Performances 5. Tools 6. Reliability 7. Conclusion Filesystem considerations ABOUT THE AUTHOR • Tristan Lelong • Embedded software engineer @ Adeneo Embedded • French, living in the Pacific northwest • Embedded software, free software, and Linux kernel enthusiast. 4 Introduction Filesystem considerations Introduction INTRODUCTION More and more embedded designs rely on smart memory chips rather than bare NAND or NOR. This presentation will start by describing: • Some context to help understand the differences between NAND and MMC • Some typical requirements found in embedded devices designs • Potential filesystems to use on MMC devices 6 Filesystem considerations Introduction INTRODUCTION Focus will then move to block filesystems. How they are supported, what feature do they advertise. To help understand how they compare, we will present some benchmarks and comparisons regarding: • Tools • Reliability • Performances 7 Block devices Filesystem considerations Block devices MMC, EMMC, SD CARD Vocabulary: • MMC: MultiMediaCard is a memory card unveiled in 1997 by SanDisk and Siemens based on NAND flash memory. -
Implementing Nfsv4 in the Enterprise: Planning and Migration Strategies
Front cover Implementing NFSv4 in the Enterprise: Planning and Migration Strategies Planning and implementation examples for AFS and DFS migrations NFSv3 to NFSv4 migration examples NFSv4 updates in AIX 5L Version 5.3 with 5300-03 Recommended Maintenance Package Gene Curylo Richard Joltes Trishali Nayar Bob Oesterlin Aniket Patel ibm.com/redbooks International Technical Support Organization Implementing NFSv4 in the Enterprise: Planning and Migration Strategies December 2005 SG24-6657-00 Note: Before using this information and the product it supports, read the information in “Notices” on page xi. First Edition (December 2005) This edition applies to Version 5, Release 3, of IBM AIX 5L (product number 5765-G03). © Copyright International Business Machines Corporation 2005. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . xi Trademarks . xii Preface . xiii The team that wrote this redbook. xiv Acknowledgments . xv Become a published author . xvi Comments welcome. xvii Part 1. Introduction . 1 Chapter 1. Introduction. 3 1.1 Overview of enterprise file systems. 4 1.2 The migration landscape today . 5 1.3 Strategic and business context . 6 1.4 Why NFSv4? . 7 1.5 The rest of this book . 8 Chapter 2. Shared file system concepts and history. 11 2.1 Characteristics of enterprise file systems . 12 2.1.1 Replication . 12 2.1.2 Migration . 12 2.1.3 Federated namespace . 13 2.1.4 Caching . 13 2.2 Enterprise file system technologies. 13 2.2.1 Sun Network File System (NFS) . 13 2.2.2 Andrew File System (AFS) . -
External Flash Filesystem for Sensor Nodes with Sparse Resources
External Flash Filesystem for Sensor Nodes with sparse Resources Stephan Lehmann Stephan Rein Clemens Gühmann stephan.lehmann@tu- stephan.rein@tu- clemens.guehmann@tu- berlin.de berlin.de berlin.de Wavelet Application Group Department of Energy and Automation Technology Chair of Electronic Measurement and Diagnostic Technology Technische Universität Berlin ABSTRACT for using just a few kilobytes of RAM, the source data must This paper describes a free filesystem for external flash mem- be accessible. Another memory intensive task concerns long- ory to be employed with low-complexity sensor nodes. The term sensor logs. system uses a standard secure digital (SD) card that can be easily connected to the serial port interface (SPI) or any A cost effective solution of this memory problem could be general input/output port of the sensor’s processor. The external flash memory. Flash memory is non-volatile and filesystem is evaluated with SD- cards used in SPI mode and therefore power failure safe. Flash memory can be obtained achieves an average random write throughput of about 40 either as raw flash or as standard flash devices such as USB- kByte/sec. For random write access throughputs larger than sticks or SD- cards. Raw flashes are pure flash chips which 400 kByte/sec are achieved. The filesystem allows for stor- do not provide a controller unit for wear levelling- that is, age of large amounts of sensor or program data and can assist a technique for prolonging the service life of the erasable more memory expensive algorithms. It requires 7 kByte of storage media - or garbage collection. -
Total Defrag™ 2009
Total Defrag™ 2009 User Manual Paragon Total Defrag™ 2009 2 User Manual CONTENTS Introduction ............................................................................................................................. 3 Key Features ............................................................................................................................ 3 Installation ............................................................................................................................... 3 Minimum System Requirements ....................................................................................................................3 Installation Procedure .....................................................................................................................................4 Basic Concepts ......................................................................................................................... 5 A Hard Disk Layout.........................................................................................................................................5 File Fragmentation...........................................................................................................................................6 Defragmentation ..............................................................................................................................................7 Interface Overview.................................................................................................................. 7 General -
MS-DOS Lecture
MS-DOS 2017 University of Babylon College of Engineering Electrical Department Learning Basics of MS-DOS Assis. Lec. Abeer Abd Alhameed | 1 MS-DOS 2017 Outcomes: By the end of this lecture, students are able to: Define the MS-DOS system Log in MS-DOS commands system Display MS-DOS information on your computer Type basic commands of MS-DOS system (view directory contents, change directory, make directory) Assis. Lec. Abeer Abd Alhameed | 2 MS-DOS 2017 Learning of MS-DOS Basics: Definition - What does Microsoft Disk Operating System (MS- DOS) mean? The Microsoft Disk Operating System (MS-DOS) is an operating system developed for PCs (personal computers) with x86 microprocessors. It was the first widely-installed operating system in personal computers. It is a command-line-based system, where all commands are entered in text form and there is no graphical user interface. The Command Prompt: When you first turn on your computer, you will see some information flash by. MS-DOS displays this information to let you know how it is configuring your computer. You can ignore it for now. When the information stops scrolling past, you'll see the following: C:\> This is called the command prompt or DOS prompt. The flashing underscore next to the command prompt is called the cursor. The cursor shows where the command you type will appear. Typing a Command: This section explains how to type a command at the command prompt and demonstrates the "Bad command or file name" message. • To type a command at the command prompt 1. Type the following at the command prompt (you can type the command in either uppercase or lowercase letters): nul If you make a typing mistake, press the BACKSPACE key to erase the mistake, and then try again. -
CS 103 Lab 1 - Linux and Virtual Machines
CS 103 Lab 1 - Linux and Virtual Machines 1 Introduction In this lab you will login to your Linux VM and write your first C/C++ program, compile it, and then execute it. 2 What you will learn In this lab you will learn the basic commands and navigation of Linux/Unix, its file system, the GNU C/C++ compiler and a few basic applications such as Gedit. Important: The Linux operating system and environment shares a vast majority of commands and methods with Unix. 3 Background Information and Notes 3.1 Software to Install on your PC Start by following the course virtual machine installation instructions found at the link below. http://cs103.usc.edu/tools-and-links/installing-course-vm/ Below is a list of recommended software you should install that will be helpful over multiple courses in CS and EE. These tools allow you to access remote servers, run GUI apps on those servers, and transfer files between your PC and those servers. For Windows: FileZilla FTP – Available from : https://software.usc.edu/index.aspx XWin-32 – Available from : https://software.usc.edu/index.aspx PuTTY – Available from : https://software.usc.edu/index.aspx For Mac: X Server: http://developer.apple.com/opensource/tools/runningx11.html Fetch FTP – Available from : https://software.usc.edu/index.aspx Reference: http://www.usc.edu/its/unix/ Last Revised: 8/29/2014 1 CS 103 Lab 1 - Linux and Virtual Machines 3.2 Getting Started with Unix1 and Accessing your Account Solaris vs. Unix vs. Linux UNIX was developed by AT&T Bell Labs in 1969. -
System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 15 SP1
SUSE Linux Enterprise Server 15 SP1 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 15 SP1 An administrator's guide for problem detection, resolution and optimization. Find how to inspect and optimize your system by means of monitoring tools and how to eciently manage resources. Also contains an overview of common problems and solutions and of additional help and documentation resources. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii 1 Available Documentation xiii