DOCSLIB.ORG

An Introduction to Disk-Based Linux File Systems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

An Introduction to Disk-Based Linux File Systems Avishay Traeger IBM Haifa Research Lab Internal Storage Course ― October 2012 v1.3 Outline The Basics The Virtual File System (VFS) File System Layout Journaling What does a disk-based file system do? Provides structure to the array of bits residing on the disk File and directory naming and hierarchy File access – open, read, write, seek, close, ... Knows how to map <file,offset> to <sector,offset> Tracks which sectors are used and which are “free” Access control Extra features (e.g., improved reliability, snapshots, compression, encryption) 3 Linux File System Types Disk (ext2/3/4, xfs, btrfs, ntfs, vfat, etc.) Network (nfs, cifs, afs, ceph, etc.) Memory (ramfs, tmpfs, etc.) Pseudo (proc, sysfs, etc.) Stackable (ecryptfs, etc.) Object store (exofs) FUSE (Filesystem in USErspace): allows developers to implement file systems in userspace (easier to develop, slower to run) ... Approximately 60 file systems currently in the Linux kernel! 4 Important Metadata Structures Superblock: on-disk metadata for entire file system Block size, pointer to fs root directory, ... Inode: on-disk metadata for a single file Inode number (unique ID), owners, timestamps, size, data block pointers, ... Dentry: metadata for a directory entry, a single component of a path (not synced to disk) File: open file structure (not synced to disk) File → Dentry → Inode → Superblock Each structure has associated operations that are implemented by each file system Note: All Linux file system implementations have the above structures in memory, but not all have superblocks and inodes on disk (especially file systems not native to Linux/Unix, like FAT). These must map on-disk structures to those in memory. 5 Directories A directory is simply a special type of file Can contain other files, directories, links, etc. Each entry has an inode number and name The file system knows how to find a file based on its inode number What are the basic steps for performing a lookup on file /foo/bar? 6 Hard Links Associate several names with one inode When creating a link, increment the inode's reference count (refcount) The inode and associated data will only be deleted when the refcount is zero Can only be used within a single file-system dentry dentry Can only point to files. This inode prevents cycles in the directory tree data Not supported by all file systems blocks 7 Symbolic Links (Symlinks) A special file that contains a file name When the kernel encounters a symlink during a pathname lookup it replaces the name of the link by its contents (the name of the target file), and restarts the pathname interpretation Can point to files on another file system Can point to any type of file (e.g., directory) Can become a dangling pointer if the target file is deleted Use more inodes than hard links (2 vs. 1) Higher overhead than hard links for resolution 8 Device Files In Linux, devices can be accessed via special files, generally found under /dev Two main types: Character: stream of bytes (keyboard, serial) Block: random access of blocks (hard disk, CD-ROM) Outline The Basics The Virtual File System (VFS) File System Layout Journaling The Virtual File System (VFS) When we have so many file systems, we need to ensure that: User programs do not need to be file-system--aware File systems don't re-implement similar functionality Solution: The VFS. A kernel layer that: Handles all system calls related to a standard Unix file system (all file systems have the same API) Handles generic activities (e.g., caching, readahead) Has generic file system “library” functions that can be used by any file system (e.g., fs/libfs.c) Each specific file system implements a set of functions (operations vectors) Object oriented programming in C 11 The Virtual File System (VFS) Application user-space kernel VFS System call Scheduler handler ext3 isofs NFS Memory Interrupt Page Cache management handler Driver Driver Driver (Disk) (CD-ROM) (Network) 12 Readahead Takes advantage of the page cache When a page is read, the VFS code may ask the file system to read the next several contiguous blocks. Hopefully, the next block read by the application will already be loaded into the page cache. Performed during: Sequential reads on files Directory reads The VFS contains the logic to perform readahead effectively 13 Example File System Operations ext3 / home etc … mnt 14 Example File System Operations ext3 / home etc … mnt avishay The VFS mount operation: 1) Calls the xfs get_sb function to read the superblock from the partition xfs 2) This function also reads the inode of the root directory mount -t xfs /dev/sdb1 /home Note that performing a lookup on 'home' would have previously invoked ext3, but now it is xfs. Any files/directories in 'home' on ext3 will now be hidden by 'home' on xfs. 15 Example File System Operations ext3 / home etc … mnt avishay cdrom foo xfs isofs mount -t xfs /dev/sdb1 /home A similar sequence of events occurs here, this time mount -t isofs /dev/hdc1 /mnt/cdrom mounting an isofs file system on a CD-ROM drive. 16 Example File System Operations ext3 / home etc … mnt avishay cdrom bar foo xfs isofs mount -t xfs /dev/sdb1 /home Lookup operations will be performed on all 3 file mount -t isofs /dev/hdc1 /mnt/cdrom systems. The copy operation cp /mnt/cdrom/foo /home/avishay/bar will read from 'foo' (isofs) and write to 'bar' (xfs). The VFS determines which file system to invoke. 17 Outline The Basics The Virtual File System (VFS) File System Layout Journaling File System Layout Some considerations: Minimize seeks between metadata and related data Minimize number of disk reads required to get to data Maximize readahead (sequential access) Recovery from disk corruption, power outage, etc. Management: fragmentation, compaction, etc. 19 Contiguous Allocation Files are allocated contiguously on the disk Space for entire file must be requested in advance Search bit map or linked list to locate a space Pros Fast sequential access Easy random access Cons External fragmentation Hard to grow files: may have to move (large) files May need compaction E A B C D 20 Linked Files (Alto) Each file is a linked list File header (like inode) points to first block on disk Each block points to the next File header Pros Can grow files dynamically File block 1 Free list is similar to a file No external fragmentation or need to move files Cons File block N Random access is horrible Even sequential access needs one seek per block Unreliable: losing one block means losing the rest 21 File Allocation Table (FAT) Table of “next pointers”, indexed by block Dentry points to 1st block of file Two copies of FAT, at the beginning of the volume Pros Faster random access Cache FAT table and traverse in memory Cons FAT table may be too large to cache - long seeks Pointers for all files are interspersed in FAT table Need full table in memory, even for one file Solution: indexed files 22 Single-Level Indexed Files User declares maximum file size A file header holds an array of pointers to disk blocks Pros Random access is fast Better metadata caching than FAT File Cons header Clumsy to grow beyond the limit Many seeks Disk blocks 23 ext2: Block Groups Boot Block Block Block Block group 0 group 1 ... group n Super Group Data Block inode inode Data block Descriptors Bitmap Bitmap Table Blocks Improved reliability Control structures are replicated Easy to recover the superblock Improved performance Reduces the distance between the inodes and related data blocks It is possible to reduce the disk head seeks during I/O on files 24 ext2: Multi-Level Indexed Files The inode contains 15 pointers: 12 direct pointers 13: 1-level indirect data 14: 2-level indirect 15: 3-level indirect 1 data Pros & Cons 2 ... In favor of small files data 13 Can grow 14 Lots of seeking 15 inode data (somewhat limited by block groups) data ext3: same on-disk format plus journal (covered later) 25 ext4/xfs/Btrfs: Extents & Trees Extent: set of logically contiguous blocks within a file that are stored contiguously on disk Single ext4 extent: up to 128MB with 4KB block size Less meta-data: Only need to remember: <1st logical block, # blocks, 1st physical block> xfs and Btrfs store extents in B-tree variants These are newer and very interesting Linux disk-based file systems and have become more “standard” 26 Log-structured File System Will be covered separately tomorrow 27 Outline The Basics The Virtual File System (VFS) File System Layout Journaling File System Corruption Some FS operations require multiple writes which may not all complete (power fail, crash) The on-disk state will be invalid on next mount Example: To write to a file, 3 main operations: 1.Write data to disk block 2.Update the free space map 3.Update pointer from inode to block With no help, detecting and recovering from errors require examining all data structures In Linux, this is done by fsck (file system check) This was acceptable in the past, but takes too long for larger file systems Journaling Journal: a special file that logs the changes destined for the file system in a circular buffer Idea: use a journal to log changes before they're committed to the file system to avoid metadata corruption Examples: JFS/JFS2, ext3/4, XFS, ReiserFS ext3 Journaling Modes Writeback: Only metadata is journaled. Data is written indepentently. Preserves file system structure and avoids corruption, but files may contain stale data (like ext2 + fast fsck).
Recommended publications
  • Copy on Write Based File Systems Performance Analysis and Implementation

    Copy on Write Based File Systems Performance Analysis and Implementation

    Copy On Write Based File Systems Performance Analysis And Implementation Sakis Kasampalis Kongens Lyngby 2010 IMM-MSC-2010-63 Technical University of Denmark Department Of Informatics Building 321, DK-2800 Kongens Lyngby, Denmark Phone +45 45253351, Fax +45 45882673 [email protected] www.imm.dtu.dk Abstract In this work I am focusing on Copy On Write based file systems. Copy On Write is used on modern file systems for providing (1) metadata and data consistency using transactional semantics, (2) cheap and instant backups using snapshots and clones. This thesis is divided into two main parts. The first part focuses on the design and performance of Copy On Write based file systems. Recent efforts aiming at creating a Copy On Write based file system are ZFS, Btrfs, ext3cow, Hammer, and LLFS. My work focuses only on ZFS and Btrfs, since they support the most advanced features. The main goals of ZFS and Btrfs are to offer a scalable, fault tolerant, and easy to administrate file system. I evaluate the performance and scalability of ZFS and Btrfs. The evaluation includes studying their design and testing their performance and scalability against a set of recommended file system benchmarks. Most computers are already based on multi-core and multiple processor architec- tures. Because of that, the need for using concurrent programming models has increased. Transactions can be very helpful for supporting concurrent program- ming models, which ensure that system updates are consistent. Unfortunately, the majority of operating systems and file systems either do not support trans- actions at all, or they simply do not expose them to the users.
  • The Kernel Report

    The Kernel Report

    The kernel report (ELC 2012 edition) Jonathan Corbet LWN.net [email protected] The Plan Look at a year's worth of kernel work ...with an eye toward the future Starting off 2011 2.6.37 released - January 4, 2011 11,446 changes, 1,276 developers VFS scalability work (inode_lock removal) Block I/O bandwidth controller PPTP support Basic pNFS support Wakeup sources What have we done since then? Since 2.6.37: Five kernel releases have been made 59,000 changes have been merged 3069 developers have contributed to the kernel 416 companies have supported kernel development February As you can see in these posts, Ralink is sending patches for the upstream rt2x00 driver for their new chipsets, and not just dumping a huge, stand-alone tarball driver on the community, as they have done in the past. This shows a huge willingness to learn how to deal with the kernel community, and they should be strongly encouraged and praised for this major change in attitude. – Greg Kroah-Hartman, February 9 Employer contributions 2.6.38-3.2 Volunteers 13.9% Wolfson Micro 1.7% Red Hat 10.9% Samsung 1.6% Intel 7.3% Google 1.6% unknown 6.9% Oracle 1.5% Novell 4.0% Microsoft 1.4% IBM 3.6% AMD 1.3% TI 3.4% Freescale 1.3% Broadcom 3.1% Fujitsu 1.1% consultants 2.2% Atheros 1.1% Nokia 1.8% Wind River 1.0% Also in February Red Hat stops releasing individual kernel patches March 2.6.38 released – March 14, 2011 (9,577 changes from 1198 developers) Per-session group scheduling dcache scalability patch set Transmit packet steering Transparent huge pages Hierarchical block I/O bandwidth controller Somebody needs to get a grip in the ARM community.
  • Rootless Containers with Podman and Fuse-Overlayfs

    Rootless Containers with Podman and Fuse-Overlayfs

    CernVM Workshop 2019 (4th June 2019) Rootless containers with Podman and fuse-overlayfs Giuseppe Scrivano @gscrivano Introduction 2 Rootless Containers • “Rootless containers refers to the ability for an unprivileged user (i.e. non-root user) to create, run and otherwise manage containers.” (https://rootlesscontaine.rs/ ) • Not just about running the container payload as an unprivileged user • Container runtime runs also as an unprivileged user 3 Don’t confuse with... • sudo podman run --user foo – Executes the process in the container as non-root – Podman and the OCI runtime still running as root • USER instruction in Dockerfile – same as above – Notably you can’t RUN dnf install ... 4 Don’t confuse with... • podman run --uidmap – Execute containers as a non-root user, using user namespaces – Most similar to rootless containers, but still requires podman and runc to run as root 5 Motivation of Rootless Containers • To mitigate potential vulnerability of container runtimes • To allow users of shared machines (e.g. HPC) to run containers without the risk of breaking other users environments • To isolate nested containers 6 Caveat: Not a panacea • Although rootless containers could mitigate these vulnerabilities, it is not a panacea , especially it is powerless against kernel (and hardware) vulnerabilities – CVE 2013-1858, CVE-2015-1328, CVE-2018-18955 • Castle approach : it should be used in conjunction with other security layers such as seccomp and SELinux 7 Podman 8 Rootless Podman Podman is a daemon-less alternative to Docker • $ alias
  • Serverless Network File Systems

    Serverless Network File Systems

    Serverless Network File Systems Thomas E. Anderson, Michael D. Dahlin, Jeanna M. Neefe, David A. Patterson, Drew S. Roselli, and Randolph Y. Wang Computer Science Division University of California at Berkeley Abstract In this paper, we propose a new paradigm for network file system design, serverless network file systems. While traditional network file systems rely on a central server machine, a serverless system utilizes workstations cooperating as peers to provide all file system services. Any machine in the system can store, cache, or control any block of data. Our approach uses this location independence, in combination with fast local area networks, to provide better performance and scalability than traditional file systems. Further, because any machine in the system can assume the responsibilities of a failed component, our serverless design also provides high availability via redundant data storage. To demonstrate our approach, we have implemented a prototype serverless network file system called xFS. Preliminary performance measurements suggest that our architecture achieves its goal of scalability. For instance, in a 32-node xFS system with 32 active clients, each client receives nearly as much read or write throughput as it would see if it were the only active client. 1. Introduction A serverless network file system distributes storage, cache, and control over cooperating workstations. This approach contrasts with traditional file systems such as Netware [Majo94], NFS [Sand85], Andrew [Howa88], and Sprite [Nels88] where a central server machine stores all data and satisfies all client cache misses. Such a central server is both a performance and reliability bottleneck. A serverless system, on the other hand, distributes control processing and data storage to achieve scalable high performance, migrates the responsibilities of failed components to the remaining machines to provide high availability, and scales gracefully to simplify system management.
  • Dm-X: Protecting Volume-Level Integrity for Cloud Volumes and Local

    Dm-X: Protecting Volume-Level Integrity for Cloud Volumes and Local

    dm-x: Protecting Volume-level Integrity for Cloud Volumes and Local Block Devices Anrin Chakraborti Bhushan Jain Jan Kasiak Stony Brook University Stony Brook University & Stony Brook University UNC, Chapel Hill Tao Zhang Donald Porter Radu Sion Stony Brook University & Stony Brook University & Stony Brook University UNC, Chapel Hill UNC, Chapel Hill ABSTRACT on Amazon’s Virtual Private Cloud (VPC) [2] (which provides The verified boot feature in recent Android devices, which strong security guarantees through network isolation) store deploys dm-verity, has been overwhelmingly successful in elim- data on untrusted storage devices residing in the public cloud. inating the extremely popular Android smart phone rooting For example, Amazon VPC file systems, object stores, and movement [25]. Unfortunately, dm-verity integrity guarantees databases reside on virtual block devices in the Amazon are read-only and do not extend to writable volumes. Elastic Block Storage (EBS). This paper introduces a new device mapper, dm-x, that This allows numerous attack vectors for a malicious cloud efficiently (fast) and reliably (metadata journaling) assures provider/untrusted software running on the server. For in- volume-level integrity for entire, writable volumes. In a direct stance, to ensure SEC-mandated assurances of end-to-end disk setup, dm-x overheads are around 6-35% over ext4 on the integrity, banks need to guarantee that the external cloud raw volume while offering additional integrity guarantees. For storage service is unable to remove log entries documenting cloud storage (Amazon EBS), dm-x overheads are negligible. financial transactions. Yet, without integrity of the storage device, a malicious cloud storage service could remove logs of a coordinated attack.
  • Enhancing the Accuracy of Synthetic File System Benchmarks Salam Farhat Nova Southeastern University, Salalimo@Gmail.Com

    Enhancing the Accuracy of Synthetic File System Benchmarks Salam Farhat Nova Southeastern University, [email protected]

    Nova Southeastern University NSUWorks CEC Theses and Dissertations College of Engineering and Computing 2017 Enhancing the Accuracy of Synthetic File System Benchmarks Salam Farhat Nova Southeastern University, [email protected] This document is a product of extensive research conducted at the Nova Southeastern University College of Engineering and Computing. For more information on research and degree programs at the NSU College of Engineering and Computing, please click here. Follow this and additional works at: https://nsuworks.nova.edu/gscis_etd Part of the Computer Sciences Commons Share Feedback About This Item NSUWorks Citation Salam Farhat. 2017. Enhancing the Accuracy of Synthetic File System Benchmarks. Doctoral dissertation. Nova Southeastern University. Retrieved from NSUWorks, College of Engineering and Computing. (1003) https://nsuworks.nova.edu/gscis_etd/1003. This Dissertation is brought to you by the College of Engineering and Computing at NSUWorks. It has been accepted for inclusion in CEC Theses and Dissertations by an authorized administrator of NSUWorks. For more information, please contact [email protected]. Enhancing the Accuracy of Synthetic File System Benchmarks by Salam Farhat A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor in Philosophy in Computer Science College of Engineering and Computing Nova Southeastern University 2017 We hereby certify that this dissertation, submitted by Salam Farhat, conforms to acceptable standards and is fully adequate in scope and quality to fulfill the dissertation requirements for the degree of Doctor of Philosophy. _____________________________________________ ________________ Gregory E. Simco, Ph.D. Date Chairperson of Dissertation Committee _____________________________________________ ________________ Sumitra Mukherjee, Ph.D. Date Dissertation Committee Member _____________________________________________ ________________ Francisco J.
  • Ext4 File System and Crash Consistency

    Ext4 File System and Crash Consistency

    1 Ext4 file system and crash consistency Changwoo Min 2 Summary of last lectures • Tools: building, exploring, and debugging Linux kernel • Core kernel infrastructure • Process management & scheduling • Interrupt & interrupt handler • Kernel synchronization • Memory management • Virtual file system • Page cache and page fault 3 Today: ext4 file system and crash consistency • File system in Linux kernel • Design considerations of a file system • History of file system • On-disk structure of Ext4 • File operations • Crash consistency 4 File system in Linux kernel User space application (ex: cp) User-space Syscalls: open, read, write, etc. Kernel-space VFS: Virtual File System Filesystems ext4 FAT32 JFFS2 Block layer Hardware Embedded Hard disk USB drive flash 5 What is a file system fundamentally? int main(int argc, char *argv[]) { int fd; char buffer[4096]; struct stat_buf; DIR *dir; struct dirent *entry; /* 1. Path name -> inode mapping */ fd = open("/home/lkp/hello.c" , O_RDONLY); /* 2. File offset -> disk block address mapping */ pread(fd, buffer, sizeof(buffer), 0); /* 3. File meta data operation */ fstat(fd, &stat_buf); printf("file size = %d\n", stat_buf.st_size); /* 4. Directory operation */ dir = opendir("/home"); entry = readdir(dir); printf("dir = %s\n", entry->d_name); return 0; } 6 Why do we care EXT4 file system? • Most widely-deployed file system • Default file system of major Linux distributions • File system used in Google data center • Default file system of Android kernel • Follows the traditional file system design 7 History of file system design 8 UFS (Unix File System) • The original UNIX file system • Design by Dennis Ritche and Ken Thompson (1974) • The first Linux file system (ext) and Minix FS has a similar layout 9 UFS (Unix File System) • Performance problem of UFS (and the first Linux file system) • Especially, long seek time between an inode and data block 10 FFS (Fast File System) • The file system of BSD UNIX • Designed by Marshall Kirk McKusick, et al.
  • XFS: There and Back ...And There Again? Slide 1 of 38

    XFS: There and Back ...And There Again? Slide 1 of 38

    XFS: There and Back.... .... and There Again? Dave Chinner <[email protected]> <[email protected]> XFS: There and Back .... and There Again? Slide 1 of 38 Overview • Story Time • Serious Things • These Days • Shiny Things • Interesting Times XFS: There and Back .... and There Again? Slide 2 of 38 Story Time • Way back in the early '90s • Storage exceeding 32 bit capacities • 64 bit CPUs, large scale MP • Hundreds of disks in a single machine • XFS: There..... Slide 3 of 38 "x" is for Undefined xFS had to support: • Fast Crash Recovery • Large File Systems • Large, Sparse Files • Large, Contiguous Files • Large Directories • Large Numbers of Files • - Scalability in the XFS File System, 1995 http://oss.sgi.com/projects/xfs/papers/xfs_usenix/index.html XFS: There..... Slide 4 of 38 The Early Years XFS: There..... Slide 5 of 38 The Early Years • Late 1994: First Release, Irix 5.3 • Mid 1996: Default FS, Irix 6.2 • Already at Version 4 • Attributes • Journalled Quotas • link counts > 64k • feature masks • • XFS: There..... Slide 6 of 38 The Early Years • • Allocation alignment to storage geometry (1997) • Unwritten extents (1998) • Version 2 directories (1999) • mkfs time configurable block size • Scalability to tens of millions of directory entries • • XFS: There..... Slide 7 of 38 What's that Linux Thing? • Feature development mostly stalled • Irix development focussed on CXFS • New team formed for Linux XFS port! • Encumberance review! • Linux was missing lots of bits XFS needed • Lot of work needed • • XFS: There and..... Slide 8 of 38 That Linux Thing? XFS: There and..... Slide 9 of 38 Light that fire! • 2000: SGI releases XFS under GPL • • 2001: First stable XFS release • • 2002: XFS merged into 2.5.36 • • JFS follows similar timeline • XFS: There and....
  • Comparing Filesystem Performance: Red Hat Enterprise Linux 6 Vs

    Comparing Filesystem Performance: Red Hat Enterprise Linux 6 Vs

    COMPARING FILE SYSTEM I/O PERFORMANCE: RED HAT ENTERPRISE LINUX 6 VS. MICROSOFT WINDOWS SERVER 2012 When choosing an operating system platform for your servers, you should know what I/O performance to expect from the operating system and file systems you select. In the Principled Technologies labs, using the IOzone file system benchmark, we compared the I/O performance of two operating systems and file system pairs, Red Hat Enterprise Linux 6 with ext4 and XFS file systems, and Microsoft Windows Server 2012 with NTFS and ReFS file systems. Our testing compared out-of-the-box configurations for each operating system, as well as tuned configurations optimized for better performance, to demonstrate how a few simple adjustments can elevate I/O performance of a file system. We found that file systems available with Red Hat Enterprise Linux 6 delivered better I/O performance than those shipped with Windows Server 2012, in both out-of- the-box and optimized configurations. With I/O performance playing such a critical role in most business applications, selecting the right file system and operating system combination is critical to help you achieve your hardware’s maximum potential. APRIL 2013 A PRINCIPLED TECHNOLOGIES TEST REPORT Commissioned by Red Hat, Inc. About file system and platform configurations While you can use IOzone to gauge disk performance, we concentrated on the file system performance of two operating systems (OSs): Red Hat Enterprise Linux 6, where we examined the ext4 and XFS file systems, and Microsoft Windows Server 2012 Datacenter Edition, where we examined NTFS and ReFS file systems.
  • CIS 191 Linux Lab Exercise

    CIS 191 Linux Lab Exercise

    CIS 191 Linux Lab Exercise Lab 9: Backup and Restore Fall 2008 Lab 9: Backup and Restore The purpose of this lab is to explore the different types of backups and methods of restoring them. We will look at three utilities often used for backups and learn the advantages and disadvantages of each. Supplies • VMWare Server 1.05 or higher • Benji VM Preconfiguration • Labs 6 and Labs 7 completed on a pristine Benji VM [root@benji /]# fdisk -l Disk /dev/sda: 5368 MB, 5368709120 bytes 255 heads, 63 sectors/track, 652 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 382 3068383+ 83 Linux /dev/sda2 383 447 522112+ 82 Linux swap / Solaris /dev/sda3 448 511 514080 83 Linux /dev/sda4 512 652 1132582+ 5 Extended /dev/sda5 512 549 305203+ 83 Linux /dev/sda6 550 556 56196 83 Linux /dev/sda7 557 581 200781 83 Linux [root@benji /]# [root@benji /]# mount /dev/sda1 on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda5 on /opt type ext3 (rw) /dev/sda3 on /var type ext3 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/sda7 on /home type ext3 (rw,usrquota) [root@benji /]# cat /etc/fstab LABEL=/1 / ext3 defaults 1 1 devpts /dev/pts devpts gid=5,mode=620 0 0 tmpfs /dev/shm tmpfs defaults 0 0 LABEL=/opt /opt ext3 defaults 1 2 proc /proc proc defaults 0 0 sysfs /sys sysfs defaults 0 0 LABEL=/var /var ext3 defaults 1 2 LABEL=SWAP-sda2 swap swap defaults 0 0 LABEL=/home /home ext3 usrquota,defaults 1 2 [root@benji /]# Forum If you get stuck on one of the steps below don’t beat your head against the wall.
  • How UNIX Organizes and Accesses Files on Disk Why File Systems

    How UNIX Organizes and Accesses Files on Disk Why File Systems

    UNIX File Systems How UNIX Organizes and Accesses Files on Disk Why File Systems • File system is a service which supports an abstract representation of the secondary storage to the OS • A file system organizes data logically for random access by the OS. • A virtual file system provides the interface between the data representation by the kernel to the user process and the data presentation to the kernel in memory. The file and directory system cache. • Because of the performance disparity between disk and CPU/memory, file system performance is the paramount issue for any OS Main memory vs. Secondary storage • Small (MB/GB) Large (GB/TB) • Expensive Cheap -2 -3 • Fast (10-6/10-7 sec) Slow (10 /10 sec) • Volatile Persistent Cannot be directly accessed • Directly accessible by CPU by CPU – Interface: (virtual) memory – Data should be first address brought into the main memory Secondary storage (disk) • A number of disks directly attached to the computer • Network attached disks accessible through a fast network - Storage Area Network (SAN) • Simple disks (IDE, SATA) have a described disk geometry. Sector size is the minimum read/write unit of data (usually 512Bytes) – Access: (#surface, #track, #sector) • Smart disks (SCSI, SAN, NAS) hide the internal disk layout using a controller type function – Access: (#sector) • Moving arm assembly (Seek) is expensive – Sequential access is x100 times faster than the random access Internal disk structure • Disk structure User Process Accessing Data • Given the file name. Get to the file’s FCB using the file system catalog (Open, Close, Set_Attribute) • The catalog maps a file name to the FCB – Checks permissions • file_handle=open(file_name): – search the catalog and bring FCB into the memory – UNIX: in-memory FCB: in-core i-node • Use the FCB to get to the desired offset within the file data: (CREATE, DELETE, SEEK, TRUNCATE) • close(file_handle): release FCB from memory Catalog Organization (Directories) • In UNIX, special files (not special device files) called directories contain information about other files.
  • Connecting the Storage System to the Solaris Host

    Connecting the Storage System to the Solaris Host

    Configuration Guide for Solaris™ Host Attachment Hitachi Virtual Storage Platform Hitachi Universal Storage Platform V/VM FASTFIND LINKS Document Organization Product Version Getting Help Contents MK-96RD632-05 Copyright © 2010 Hitachi, Ltd., all rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi, Ltd. (hereinafter referred to as “Hitachi”) and Hitachi Data Systems Corporation (hereinafter referred to as “Hitachi Data Systems”). Hitachi Data Systems reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new and/or revised information becomes available, this entire document will be updated and distributed to all registered users. All of the features described in this document may not be currently available. Refer to the most recent product announcement or contact your local Hitachi Data Systems sales office for information about feature and product availability. Notice: Hitachi Data Systems products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems agreement(s). The use of Hitachi Data Systems products is governed by the terms of your agreement(s) with Hitachi Data Systems. Hitachi is a registered trademark of Hitachi, Ltd. in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd.