DOCSLIB.ORG

A Thin Server Management System Using Bittorrent Chris Mceniry – Sony Computer Entertainment America

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Thin Server Management System Using Bittorrent Chris Mceniry – Sony Computer Entertainment America Moobi: A Thin Server Management System Using BitTorrent Chris McEniry – Sony Computer Entertainment America ABSTRACT We describe a tool that provides a method for running dataless caching clients – a hybrid combination of imaging systems with traditional diskless nodes. Unlike imaging systems, it is a single boot to get to a running system; unlike diskless systems, it is more robust and scalable as it does not continuously depend on central servers. The tool, Moobi, uses the peer-to-peer protocol BitTorrent to provide efficient distribution of the image cache, and combines standard diskless tools to provide the basis for the running system. Moobi makes it possible to run large installations of ‘‘thin server’’ farms. Introduction system for these setups must account and check for ev- ery piece individually to validate the consistency of the Diskless clients are appealing for large clusters environment. of similar systems performing similar functions since they limit the capability for local configurations and, Truly diskless clients do not have this problem. more specifically, local misconfigurations on each sys- Nothing is locally maintained, and a single instance, tem. This provides a more uniform environment and typically of an NFS server, is directly used for the en- allows for the management of a large number of sys- vironment. This provides the consistency, but loses in tems with fewer resources. The key for these systems performance: diskless clients are very hard to scale to is not so much that they are diskless in that they are 100s of nodes. This limitation on scaling is due to dataless – there is nothing which is inherently tied to their reliance on the large file servers that support the the nodes. In the desktop realm, these are typically re- diskless nodes. An installed environment suffers a ferred to as thin clients. In the server realm, ‘‘throw similar scaling issue whenever massive deployments away’’ and ‘‘field replaceable units(FRUs)’’ are some have to happen in short order – initial spin ups, disas- common terms. ter recovery, or massive updates. Diskless client envi- ronments have a continuous scaling issue, while the The difference between diskless and dataless installed environments have ‘‘impulse’’ scaling issues. hinges upon drive cost, application robustness over drive failure rates, and the need for local working disk. Moobi combines the consistency of diskless sys- The size of drives has increased significantly, especially tems with the efficiency of imaged or installed sys- when compared to the size of operating systems and ap- tems. It achieves this by dividing the operating system plications bases. Pinheiro, et al. [1] demonstrated that image into a small configurable portion – the root or annual hard drive failure rates were between 2% and /etc – and a large fixed image – /usr – and by caching 10% depending on age and usage; even with 2%, in a and sharing the large image via BitTorrent. The fixed large population of machines, there is an inherent need image is maintained as a whole instance, so only one for applications to account for downed nodes and item needs to be validated against the source. Every thereby make drive failure moot. In addition, a local time a node boots, or as often as is necessary for audit- drive provides a convenient location for cached data or ing purposes, it can scan the cache and downloads and temporary work space, and so they are included in most updates the cache with any new image. clusters regardless of management methods. These rea- BitTorrent is used for validating and updating the sons push commodity servers towards having local stor- image cache. Due to its nature, these two operations are age and treating it as a large cache or work disk, but not inherently connected. In order to download and update, as a true data disk. it must validate. It sees updating as the only logical fol- Since these systems have drives, they also have a lowup after validating. In addition, BitTorrent allows unique local instance of the operating system. Whether for an update mechanism which does not require large this operating system is installed via an imaging system, monolithic file servers; thus it alleviates the bulk of the or installed via an automated installation mechanism is scaling issues for large installations. In addition, BitTor- largely irrelevant since the result is an individual system rent allows for multiple systems to distribute the image instance with a unique instance of every part of that sys- without much additional logic. This makes it easier to tem. This individual identity is necessary for certain make the system most robust overall. items – hostname, ssh keys – but is undesirable for other The mechanisms used in Moobi provide addi- items – /usr software. Any configuration management tional availability for applications running on Moobi 21st Large Installation System Administration Conference (LISA ’07) 253 Moobi: A Thin Server Management System Using BitTorrent McEniry nodes. Not only does Moobi reduce the MTTR, but it operating systems, and also have a long history of per- also provides several hooks for system hardening. Peri- formance issues with a central server [7]. Traugott and odic image checking provides a level of confidence in Huddleston [8] highlight Sun Autoclient’s CacheFS the sanctity of the software being used. In addition, the features to reduce this. Moobi reduces these issues to static file system images, such as the software image occurring only at boot time, and allows for redundant /usr, can be mounted with read-only permissions. Appli- boot servers with little or no synchronization between cation stacks could be wrapped up in a similar manner them. to provide extra confidence in the stack. BitTorrent is heavily in use in distributing large Going to a hybrid solution also acts as a stepping software images to large audiences. Many linux distri- stone. Most organizations grow organically from indi- butions, live CDs, large software applications, and vidual system instances to dataless systems. Hard VMWare appliance images [9] are distributed using drives tend to remain an item in new systems, and are BitTorrent. There are discussions of integrating Bit- not removed from repurposed systems. Torrent with linux distributions’ installation or update systems, but this discussion has produced little. Related Work Recently, SystemImager [10] of the System Instal- Given Moobi’s hybrid nature, similar work falls lation Suite has incorporated BitTorrent as a distribution into either the imaging and installation category or in- method for its imaging system. Moobi and SystemIm- to the diskless client category, yet neither category ager run into many similar obstacles, especially when completely describes it. Unlike imaging systems, it is handling image versus host specific overlays, and when a single boot to get to a running system; unlike disk- attempting to achieve high performance and scalability. less systems, it is less fragile as it does not continuous- However, SystemImager still views the many systems ly depend on central servers. as single instances and can fall into divergent configura- Several commercial and noncommercial imaging tions. In addition, SystemImager has a two boot imaging systems exist to date. Norton Ghost [2] is the industry operation – one boot for the imaging, and another boot standard for commercial system imaging software. to start the imaged system for normal operation. Several noncommercial systems provide similar fea- Moobi Design tures to Ghost, and have been able to address many of the distribution issues. Frisbee [3], in particular, has Moobi Basics been shown to have very good image distribution per- Several aspects are assumed for the Moobi image formance [4]. Partition Image [5] provides a wide building and deployment discussion: how Moobi lever- range of image support, but is not very efficient in its ages the linux boot process, how Moobi distributes its distribution. own software, how additional configuration information Ty p i c a l system imaging packages use one of two is passed to identical nodes, and host specific overlays. network mechanisms for the transporting of the golden First, Moobi intercepts the normal linux boot image to the target machine: a unicast transfer, or a mul- process to prepare a specialized environment prior to ticast or broadcast blast. Unicast transfers can be very handing off to the standard init process. expensive on the boot servers. Broadcast is very stress- Almesberger [11] describes the linux boot process ing on the network. Multicast requires advanced syn- in great detail. The relevant portions for this discussion chronization techniques and places additional con- are summarized here. With loadable modules being as straints on how and when nodes can come online. In prominent as they are, almost all recent linux boots re- contrast, a BitTorrent swarm limits server load, limits quire the use of the initrd. Typically, the initrd is a pseu- the scope of network usage to the ports involved, and do-root file system which contains the kernel module automatically handles synchronization – if a BitTorrent utilities as well as any modules necessary for the system client is late to the swarm, its peers will catch it up. be able to mount the true root file system. If this is a lo- All imaging systems require some mechanism to cal disk, disk drivers and file system drives are loaded. If address host specific file overlays. Typically, this is done this is a network file system, the NFS modules are load- by just copying down the necessary files from a central ed. Once the kernel is done with its primary initializa- system. Ghost in particular is very good at performing tion, it mounts the initrd as a ramdisk and executes the post imaging instance fix-ups to reduce unique identifier linuxrc script contained therein. Typically, the linuxrc conflict without needing a central software repository. script is simplified to the point of just loading the mod- However, it does not address non-windows operating ules necessary and then hands off execution to init.
Load more

Recommended publications
  • Oracle® ZFS Storage Appliance Security Guide, Release OS8.6.X
    ® Oracle ZFS Storage Appliance Security Guide, Release OS8.6.x Part No: E76480-01 September 2016 Oracle ZFS Storage Appliance Security Guide, Release OS8.6.x Part No: E76480-01 Copyright © 2014, 2016, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs.
    [Show full text]
  • Virtfs—A Virtualization Aware File System Pass-Through
    VirtFS—A virtualization aware File System pass-through Venkateswararao Jujjuri Eric Van Hensbergen Anthony Liguori IBM Linux Technology Center IBM Research Austin IBM Linux Technology Center [email protected] [email protected] [email protected] Badari Pulavarty IBM Linux Technology Center [email protected] Abstract operations into block device operations and then again into host file system operations. This paper describes the design and implementation of In addition to performance improvements over a tradi- a paravirtualized file system interface for Linux in the tional virtual block device, exposing guest file system KVM environment. Today’s solution of sharing host activity to the hypervisor provides greater insight to the files on the guest through generic network file systems hypervisor about the workload the guest is running. This like NFS and CIFS suffer from major performance and allows the hypervisor to make more intelligent decisions feature deficiencies as these protocols are not designed with respect to I/O caching and creates new opportuni- or optimized for virtualization. To address the needs of ties for hypervisor-based services like de-duplification. the virtualization paradigm, in this paper we are intro- ducing a new paravirtualized file system called VirtFS. In Section 2 of this paper, we explore more details about This new file system is currently under development and the motivating factors for paravirtualizing the file sys- is being built using QEMU, KVM, VirtIO technologies tem layer. In Section 3, we introduce the VirtFS design and 9P2000.L protocol. including an overview of the 9P protocol, which VirtFS is based on, along with a set of extensions introduced for greater Linux guest compatibility.
    [Show full text]
  • Managing Network File Systems in Oracle® Solaris 11.4
    Managing Network File Systems in ® Oracle Solaris 11.4 Part No: E61004 August 2021 Managing Network File Systems in Oracle Solaris 11.4 Part No: E61004 Copyright © 2002, 2021, Oracle and/or its affiliates. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software" or "commercial computer software documentation" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract.
    [Show full text]
  • Chapter 15: File System Internals
    Chapter 15: File System Internals Operating System Concepts – 10th dition Silberschatz, Galvin and Gagne ©2018 Chapter 15: File System Internals File Systems File-System Mounting Partitions and Mounting File Sharing Virtual File Systems Remote File Systems Consistency Semantics NFS Operating System Concepts – 10th dition 1!"2 Silberschatz, Galvin and Gagne ©2018 Objectives Delve into the details of file systems and their implementation Explore "ooting and file sharing Describe remote file systems, using NFS as an example Operating System Concepts – 10th dition 1!"# Silberschatz, Galvin and Gagne ©2018 File System $eneral-purpose computers can have multiple storage devices Devices can "e sliced into partitions, %hich hold volumes Volumes can span multiple partitions Each volume usually formatted into a file system # of file systems varies, typically dozens available to choose from Typical storage device organization) Operating System Concepts – 10th dition 1!"$ Silberschatz, Galvin and Gagne ©2018 Example Mount Points and File Systems - Solaris Operating System Concepts – 10th dition 1!"! Silberschatz, Galvin and Gagne ©2018 Partitions and Mounting Partition can be a volume containing a file system +* cooked-) or ra& – just a sequence of "loc,s %ith no file system Boot bloc, can point to boot volume or "oot loader set of "loc,s that contain enough code to ,now how to load the ,ernel from the file system 3r a boot management program for multi-os booting 'oot partition contains the 3S, other partitions can hold other
    [Show full text]
  • File-System Implementation
    CHAPTER File-System 11 Implementation As we saw in Chapter 10, the ®le system provides the mechanism for on-line storage and access to ®le contents, including data and programs. The ®le system resides permanently on secondary storage, which is designed to hold a large amount of data permanently. This chapter is primarily concerned with issues surrounding ®le storage and access on the most common secondary-storage medium, the disk. We explore ways to structure ®le use, to allocate disk space, to recover freed space, to track the locations of data, and to interface other parts of the operating system to secondary storage. Performance issues are considered throughout the chapter. Bibliographical Notes The MS-DOS FAT system is explained in[Norton and Wilton (1988)]. The internals of the BSD UNIX system are covered in full in[McKusick and Neville- Neil (2005)]. Details concerning ®le systems for Linux can be found in[Love (2010)]. The Google ®le system is described in[Ghemawat et al. (2003)]. FUSE can be found at http://fuse.sourceforge.net. Log-structured ®le organizations for enhancing both performance and consistency are discussed in[Rosenblum and Ousterhout (1991)], [Seltzer et al. (1993)], and[Seltzer et al. (1995)]. Algorithms such as balanced trees (and much more) are covered by[Knuth (1998)] and [Cormen et al. (2009)].[Silvers (2000)] discusses implementing the page cache in the NetBSD operating system. The ZFS source code for space maps can be found at http://src.opensolaris.org/source/xref/onnv/onnv- gate/usr/src/uts/common/fs/zfs/space map.c. The network ®le system (NFS) is discussed in[Callaghan (2000)].
    [Show full text]
  • Comparative Analysis of Distributed and Parallel File Systems' Internal Techniques
    Comparative Analysis of Distributed and Parallel File Systems’ Internal Techniques Viacheslav Dubeyko Content 1 TERMINOLOGY AND ABBREVIATIONS ................................................................................ 4 2 INTRODUCTION......................................................................................................................... 5 3 COMPARATIVE ANALYSIS METHODOLOGY ....................................................................... 5 4 FILE SYSTEM FEATURES CLASSIFICATION ........................................................................ 5 4.1 Distributed File Systems ............................................................................................................................ 6 4.1.1 HDFS ..................................................................................................................................................... 6 4.1.2 GFS (Google File System) ....................................................................................................................... 7 4.1.3 InterMezzo ............................................................................................................................................ 9 4.1.4 CodA .................................................................................................................................................... 10 4.1.5 Ceph.................................................................................................................................................... 12 4.1.6 DDFS ..................................................................................................................................................
    [Show full text]
  • Zfs-Ascalabledistributedfilesystemusingobjectdisks
    zFS-AScalableDistributedFileSystemUsingObjectDisks Ohad Rodeh Avi Teperman [email protected] [email protected] IBM Labs, Haifa University, Mount Carmel, Haifa 31905, Israel. Abstract retrieves the data block from the remote machine. zFS also uses distributed transactions and leases, instead of group- zFS is a research project aimed at building a decentral- communication and clustering software. We intend to test ized file system that distributes all aspects of file and stor- and show the effectiveness of these two features in our pro- age management over a set of cooperating machines inter- totype. connected by a high-speed network. zFS is designed to be zFS has six components: a Front End (FE), a Cooper- a file system that scales from a few networked computers to ative Cache (Cache), a File Manager (FMGR), a Lease several thousand machines and to be built from commodity Manager (LMGR), a Transaction Server (TSVR), and an off-the-shelf components. Object Store (OSD). These components work together to The two most prominent features of zFS are its coop- provide applications/users with a distributed file system. erative cache and distributed transactions. zFS integrates The design of zFS addresses, and is influenced by, issues the memory of all participating machines into one coher- of fault tolerance, security and backup/mirroring. How- ent cache. Thus, instead of going to the disk for a block ever, in this article, we focus on the zFS high-level archi- of data already in one of the machine memories, zFS re- tecture and briefly describe zFS’s fault tolerance character- trieves the data block from the remote machine.
    [Show full text]
  • Sun Storedge™ QFS and Sun Storedge SAM-FS Software Installation and Configuration Guide
    Sun StorEdge™ QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide Release 4.2 Sun Microsystems, Inc. www.sun.com Part No. 817-7722-10 September 2004, Revision A Submit comments about this document at: http://www.sun.com/hwdocs/feedback Copyright 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual property rights relating to technology that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at http://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and in other countries. This document and the product to which it pertains are distributed under licenses restricting their use, copying, distribution, and decompilation. No part of the product or of this document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd. Sun, Sun Microsystems, the Sun logo, AnswerBook2, docs.sun.com, Solaris, SunOS, SunSolve, Java, JavaScript, Solstice DiskSuite, and StorEdge are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and in other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc.
    [Show full text]
  • Distributed Storage
    Distributed and Federated Storage How to store things… in… many places... (maybe) CS2510 Presented by: wilkie [email protected] University of Pittsburgh Recommended Reading (or Skimming) • NFS: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.473 • WAFL: https://dl.acm.org/citation.cfm?id=1267093 • Hierarchical File Systems are Dead (Margo Seltzer, 2009): https://www.eecs.harvard.edu/margo/papers/hotos09/paper.pdf • Chord (Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan, 2001): https://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf • Kademlia (Petar Maymounkov, David Mazières, 2002): https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf • BitTorrent Overview: http://web.cs.ucla.edu/classes/cs217/05BitTorrent.pdf • IPFS (Juan Benet, 2014): https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ ipfs.draft3.pdf (served via IPFS, neat) Network File System NFS: A Traditional and Classic Distributed File System Problem • Storage is cheap. • YES. This is a problem in a classical sense. • People are storing more stuff and want very strong storage guarantees. • Networked (web) applications are global and people want strong availability and stable speed/performance (wherever in the world they are.) Yikes! • More data == Greater probability of failure • We want consistency (correct, up-to-date data) • We want availability (when we need it) • We want partition tolerance (even in the presence of downtime) • Oh. Hmm. Well, heck. • That’s hard (technically impossible) so what can we do? Lightning Round: Distributed Storage • Network File System (NFS) • We will gloss over details, here, Unreliable but the papers are definitely worth a read. • It invented the Virtual File System (VFS) • Basically, though, it is an early attempt to investigate the trade-offs for client/server file consistency Most Reliable?? NFS System Model • Each client connects directly to the server.
    [Show full text]
  • Ops Center Protector File System Support Matrix
    Hitachi Ops Center Protector v7.0 Support Matrix: File Systems Introduction: Agent-Based and Agentless Protection Protector can provide consistent protection either with or without the use of a host agent. • Agent-based protection performs the application interaction required to provide application-aware consistency. • Agentless protection ensures the application data is protected on disk and therefore provides crash consistency. Pre-execution script- and post-execution scripts can be called as part of a policy to enable application aware consistency. This support matrix describes the environments supported by either method of protection. Agent-Based Protection Agent-based protection provides application consistent protection support with the following operating systems and configurations. Operating System(1) Supported Protection Types Host Based Storage Based Gen 1 Gen 2 Hitachi Repository Repository, Block Gen 2 HCP, Amazon S3 Microsoft Windows Windows Server 2012 (64-bit) Windows Server 2012 R2 (64-bit) Windows Server 2016 (64-bit) (2) (NTFS, ReFS) (NTFS, ReFS) (NTFS, ReFS) Windows Server 2019(2) Microsoft Windows (Desktop) Windows 8 and 8.1 Windows 10 (NTFS) (NTFS) Linux (3)(4) RHEL 6 x64 (6.3 and newer) RHEL 7 x64 (7.0 and newer) RHEL 8 x64 (8.0 and 8.1 Only) OEL 6 x64 (6.3 and newer) (EXT3, EXT4, (EXT3, EXT4, OEL 7 x64 (7.0 and newer) (EXT3, LVM, NFS, LVM, NFS, OEL 8 x64 (8.0 and 8.1 Only) EXT4,LVM,ASM) SUSE 11 x64 (11.3 and newer) XFS,ASM) XFS,ASM)) SUSE 12 x64 (12.0 and newer) SUSE 15 x64(15.0 SP1 Only) NFS = network file system,
    [Show full text]
  • COS 318: Operating Systems Journaling, NFS and WAFL
    COS 318: Operating Systems Journaling, NFS and WAFL Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) Topics ◆ Journaling and LFS ◆ Network File System ◆ NetApp File System 2 Log-structured File System (LFS) ◆ Structure the entire file system as a log with segments ! A segment has i-nodes, indirect blocks, and data blocks ! An i-node map to map i-node number to i-node locations ! All writes are sequential ◆ Issues ! There will be holes when deleting files ! Need garbage collection to get rid of holes ! Read performance? ◆ Goal is to improve write performance ! Not to confuse with the log for transactions/journaling ! Also useful for write and wear-leveling with NAND Flash i P D i P D i P D i P D i P D Unused Inode Log structured map Revisit Implementation of Transactions ◆ BeginTransaction ! Start using a “write-ahead” log on disk ! Log all updates ◆ Commit ! Write “commit” at the end of the log ! Then “write-behind” to disk by writing updates to disk ! Clear the log ◆ Rollback ! Clear the log ◆ Crash recovery ! If there is no “commit” in the log, do nothing ! If there is “commit,” replay the log and clear the log ◆ Issues ! All updates on the log must be idempotent ! Each transaction has an Id or TID ! Must have a way to confirm that a disk write completes 4 Journaling File System ◆ Example: Append a data block to a file on disk ! Allocate disk blocks for data and i-node (update bitmap) ! Update i-node and data blocks ◆ Journaling all updates ! Execute the following transaction:
    [Show full text]
  • Chapter 11: File System Implementation
    Chapter 11: File System Implementation File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems NFS Example: WAFL File System Objectives To describe the details of implementing local file systems and directory structures To describe the implementation of remote file systems To discuss block allocation and free-block algorithms and trade-offs File-System Structure File structure Logical storage unit Collection of related information File system resides on secondary storage (disks) File system organized into layers File control block – storage structure consisting of information about a file Layered File System A Typical File Control Block In-Memory File System Structures The following figure illustrates the necessary file system structures provided by the operating systems. Figure 12-3(a) refers to opening a file. Figure 12-3(b) refers to reading a file. In-Memory File System Structures Virtual File Systems Virtual File Systems (VFS) provide an object-oriented way of implementing file systems. VFS allows the same system call interface (the API) to be used for different types of file systems. The API is to the VFS interface, rather than any specific type of file system. Schematic View of Virtual File System Directory Implementation Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute Hash Table – linear list with hash data structure. decreases
    [Show full text]