DOCSLIB.ORG

The Massive Deployment Tools: DRBL and Clonezilla

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Massive Deployment Tools: DRBL and Clonezilla TheThe MassiveMassive DeploymentDeployment Tools:Tools: DRBLDRBL andand ClonezillaClonezilla StevenSteven Shiau,Shiau, JazzJazz Wang,Wang, CeasarCeasar Sun,Sun, ThomasThomas TsaiTsai NCHC,NCHC, TaiwanTaiwan Q1,Q1, 20092009 Outline Introduction to DRBL/Clonezilla About DRBL/Clonezilla Users in Taiwan and Worldwide Related Activities Success Stories Testimonials Demo DRBL – Diskless Linux client on a virtual machine Q&A Free Software Lab Free software projects developed by NCHC's Free Software Lab Our goal is to create a Free Software Smart Classroom (FSSC) based on our HPC experience DRBL, Clonezilla, DRBL- WinRoll, Tux2Live, PartClone Multicast VNC Access Grid, eLearning Cluster Computing Grid Computing DRBL/Clonezilla DRBL Clonezilla Diskless Remote Boot in Linux Provides a “diskless” A partitioning and disk and/or “systemless” cloning utility similar to environment for client Ghost and True image machines DRBL Saves Money No HD required in the client machine(s) Truly “Free” Software--freedom to improve upon and free to use Saves Time All programs are installed on a single server One command to Shutdown/Reboot/Power On/Off all client machines Makes Life Easier No client disk failure Client machines are plug-and-play Computer classroom environment is extremely flexible GNU/Linux & MS Windows co-exist DRBL vs. LTSP What's the difference between DRBL and LTSP?: DRBL: Distributed Resource vs. LTSP: Centralized Resource DRBL: Powerful Client vs. LTSP: Thin Client DRBL allows 3D gaming and video playback in the client; LTSP does not Why DRBL? DRBL Special Features: Clonezilla Server Edition (SE) AoE service available using diskless MS Windows XP A single command is required to switch the client environment DRBL, FreeDOS, OS in local HD, memtest... Linux network installation Not suitable for older machines and/or thin clients //NOTE// From version 5 (released '07), fat client mechanism is supported in LTSP “Bare Metal” Backup and Restore “Bare metal” restore -> A technique in the field of data recovery and restoration where the backed-up data is available in a form which allows one to restore a computer system from "bare metal” (i.e. without any requirements as to previously installed software or operating systems). – Wikipedia Image-based backup & restore is different from file-based (e.g. 'cp', 'tar', 'rsync'...) Clonezilla Features Free (GPL) Software File systems supported: Ext2/3, ReiserFS, Reiser4, XFS, jfs, HFS+, FAT and NTFS Linux Logical Volume Manager 2 (LVM2) stackable block device is supported Multicast supported in Clonezilla Server Edition (SE) Clonezilla Features Saves and restores partitions and entire disks If file system is supported, only the used blocks in the HD are saved and restored. For unsupported file systems, sector-to-sector copying is done via dd. DRBL-Winroll: The hostname, group, and SID of the cloned MS Windows machine can be changed automatically Clonezilla Based on existing software: Partimage ntfsclone dd updcast DRBL (used for massive cloning) Partclone (Developed @ the NCHC, '07) and more..... Partclone http://www.partclone.org Developed by Thomas Tsai and Jazz Wang Only the used blocks are saved and restored Supports Ext2, Ext3, ReiserFS, Reiser4, XFS, HFS+, NTFS & FAT Will support for ZFS and BTRFS in the future! Two Clonezilla Types Clonezila Server Clonezilla Live Edition (SE) Single machine Massive cloning backup Network boot (PXE) Based on DRBL server is DebianLive (live- required helper and live initramfs) CD/USB flash drive is required Also available in Clonezilla- SysRescCD, created by Spiros Georgaras DRBL (Includes Clonezilla SE) server switch client nodes pxe/etherboot pxe/etherboot DHCP IP 192.168.0.1 192.168.0.40 TFTP kernel boot boot NFS file system /, /usr, /home ... /, /usr, /home ... NIS account user login user login DRBL Schematic public ip NIC 1 server NIC 2 NIC 3 NIC 4 private ip private ip private ip switch 1 switch 2 switch 3 node001 node013 node014 node027 node028 node040 ~ ~ ~ DRBL Clonezilla Live On CD, USB flash drive, or PXE server Languages: Traditional Chinese, Simplified Chinese (thanks to Zhiqiang Zhang and Liang Qi), English, French (thanks to Jean-Francois Nifenecker), Japanese (thanks to Akira Yoshiyama and Annie Wei), Italian (thanks to Gianfranco Gentili), Spanish (thanks Juan Ramón Martínez). General Purpose Use- Save or restore interactively Special Purpose Use- Recovery CD/USB drive Clonezilla image embedded in CD or USB drive One command restores the entire image! One key restores the entire image! Completely Customizable! Make Clonezilla Live work for you! Clonezilla Live DRBL/Clonezilla Use In Taiwan Public Sector: Primary/High schools 295 Universities 95 Local governments 35 Hospitals 3 TV stations 1 SMBs 124 Misc. 2 Known sites/users >500!!!>500!!! DRBL/Clonezilla Use Worldwide 西班牙 中國 南非 澳洲 >7>700,00000,000 DRBL/Clonezilla Downloads!!! Related Activities Ubuntu Live Conference Portland, Oregon, USA, July 23rd, 2007 Clonezilla: It's Better Than Ghost! Speaker: Kristian Hermansen, Security Researcher/Developer, Cisco Systems, Inc. http://www.ubuntulive.com/cs/ubuntu/view/e_se ss/12510 LinuxWorld Conference & Expo San Francisco, USA, Aug, 2007 Dell booth Massive Deployments Speaker: Torsten Spindler, Canonical Related Activities Libre Software Meeting (LSM) 2008 Mont-de-Marsan, France, July 1st~5th, 2008 “Clonezilla Live, the “Bare Metal” Backup and Recovery Tool“ Speaker: Steven Shiau http://2008.rmll.info/Clonezilla-live-the-bare- metal.html?lang=en LinuxWorld Conference & Expo San Francisco, USA, Aug, 2008 .org Pavilion (booth #: .org 3) BoF: Clonezilla Hands-on Lab: The Universal “Bare Metal” Backup and Recovery Tool for Linux, Mac OS and MS Windows Ubuntu Live, USA, Jul/07 LinuxWorld; “Massive Deployments;” Aug/07 International Free Software Contest DRBL is the winner in the 'Public Sector Software' category at Les Trophees du Libre 2007 in France Finals and awards ceremony: Nov. 29th, 2007 http://www.freesoftwareawards.com or http://tropheesdulibre.com/?lang=en Intrnl. Free Software Contest, France, Nov/07 LSM 2008, France LinuxWorld, San Fran, '08 Aug 4-7, 2008 One of the largest Linux expos. .org Pavilion (booth #: .org 3) BoF: “Clonezilla Hands-on Lab: The Universal “Bare Metal” Backup and Recovery Tool for Linux, Mac OS and MS Windows” (Aug. 6th, '08) National Applied Research Laboratories Award DRBL/Clonezilla team is the winner in the category 'Technology Development' for 2008 Success Stories Da-Feng Elementary School, Taipei, Taiwan DRBL/Clonezilla is used to manage computer classroom Special Report: Linux in Education, 2006 on distrowatch.com DRBL/Clonezilla press conference @ Da- Feng Elementary School 07/04/07 source: distrowatch.com Success Stories Hualien County, Taiwan Used Clonezilla Server Edition (SE) to clone GNU/Linux 145 schools (131 Primary/Jr.; 14 Senior High) use as of June '08 Letter of Commendation from Hualien County Government Clonezilla Used in Taiwan's “National PCs” Source: De-Wen Huang Testimonials Kristian Hermansen Cisco Systems, Boston, MA, Sept. '07 Cloned a 5GB image to 64 machines in 5 under minutes!!! "I used Clonezilla in the design of our Cisco Computational Cloud Cluster to multicast a 5GB disk image to 64 machines simultaneously and all in under five minutes! Amazing! Thanks Clonezilla!" Testimonials Barny Sanchez Information Systems Security, Southbridge, Massachusetts Cloned 1,084 systems using DRBL (Clonezilla SE) "I've used DRBL to clone 1,084 systems so far! It was simple! All I had to do was divide each system into groups of 80-100 PCs and then use multicasting to do the cloning. It took anywhere from 16-38 minutes to clone each system. The images of various operating systems averaged 1GB in size. DRBL has reduced the recovery/cloning factor by more than 500% as compared to the commercial solution I used previously! You can imagine how happy my project managers are!" What's New Since SEAIP 2007 ? DRBL Supports newer GNU/Linux versions including Ubuntu 8.04/8.10, Fedora 9, Mandriva 2008.1, and OpenSuSE 11.0 Provides AoE booting mechanism in support of diskless MS Windows XP Clonezilla Live Easier to create a recovery ISO/zip Clonezilla Live Japanese, Simplified Chinese and French locales were added Serial console is supported Demo • Diskless Linux client on a virtual machine – Server: DRBL 1.9.3-30 running on Ubuntu Linux 8.10 – Client: Virtualbox 2.0.4 as a client machine, no local hard drive – Boot client via PXE (network booting) QuestionsQuestions ?? Great! ?????.
Load more

Recommended publications
  • Filesystem Considerations for Embedded Devices ELC2015 03/25/15
    Filesystem considerations for embedded devices ELC2015 03/25/15 Tristan Lelong Senior embedded software engineer Filesystem considerations ABSTRACT The goal of this presentation is to answer a question asked by several customers: which filesystem should you use within your embedded design’s eMMC/SDCard? These storage devices use a standard block interface, compatible with traditional filesystems, but constraints are not those of desktop PC environments. EXT2/3/4, BTRFS, F2FS are the first of many solutions which come to mind, but how do they all compare? Typical queries include performance, longevity, tools availability, support, and power loss robustness. This presentation will not dive into implementation details but will instead summarize provided answers with the help of various figures and meaningful test results. 2 TABLE OF CONTENTS 1. Introduction 2. Block devices 3. Available filesystems 4. Performances 5. Tools 6. Reliability 7. Conclusion Filesystem considerations ABOUT THE AUTHOR • Tristan Lelong • Embedded software engineer @ Adeneo Embedded • French, living in the Pacific northwest • Embedded software, free software, and Linux kernel enthusiast. 4 Introduction Filesystem considerations Introduction INTRODUCTION More and more embedded designs rely on smart memory chips rather than bare NAND or NOR. This presentation will start by describing: • Some context to help understand the differences between NAND and MMC • Some typical requirements found in embedded devices designs • Potential filesystems to use on MMC devices 6 Filesystem considerations Introduction INTRODUCTION Focus will then move to block filesystems. How they are supported, what feature do they advertise. To help understand how they compare, we will present some benchmarks and comparisons regarding: • Tools • Reliability • Performances 7 Block devices Filesystem considerations Block devices MMC, EMMC, SD CARD Vocabulary: • MMC: MultiMediaCard is a memory card unveiled in 1997 by SanDisk and Siemens based on NAND flash memory.
    [Show full text]
  • Netinfo 2009-06-11 Netinfo 2009-06-11
    Netinfo 2009-06-11 Netinfo 2009-06-11 Microsoft släppte 2009-06-09 tio uppdateringar som täpper till 31 stycken säkerhetshål i bland annat Windows, Internet Explorer, Word, Excel, Windows Search. 18 av buggfixarna är märkta som kritiska och elva av dem är märkta som viktiga, uppdateringarna finns för både servrar och arbetsstationer. Säkerhetsuppdateringarna finns tillgängliga på Windows Update. Den viktigaste säkerhetsuppdateringen av de som släpptes är den för Internet Explorer 8. Netinfo 2009-06-11 Security Updates available for Adobe Reader and Acrobat Release date: June 9, 2009 Affected software versions Adobe Reader 9.1.1 and earlier versions Adobe Acrobat Standard, Pro, and Pro Extended 9.1.1 and earlier versions Severity rating Adobe categorizes this as a critical update and recommends that users apply the update for their product installations. These vulnerabilities would cause the application to crash and could potentially allow an attacker to take control of the affected system. Netinfo 2009-06-11 SystemRescueCd Description: SystemRescueCd is a Linux system on a bootable CD-ROM for repairing your system and recovering your data after a crash. It aims to provide an easy way to carry out admin tasks on your computer, such as creating and editing the partitions of the hard disk. It contains a lot of system tools (parted, partimage, fstools, ...) and basic tools (editors, midnight commander, network tools). It is very easy to use: just boot the CDROM. The kernel supports most of the important file systems (ext2/ext3/ext4, reiserfs, reiser4, btrfs, xfs, jfs, vfat, ntfs, iso9660), as well as network filesystems (samba and nfs).
    [Show full text]
  • Zack's Kernel News
    KERNEL NEWS ZACK’S KERNEL NEWS ReiserFS Turmoil Their longer term plan, Alexander Multiport Card driver, again naming In light of recent events surrounding says, depends on what happens with himself the maintainer. Hans Reiser (http:// www. linux-maga- Hans. If Hans is released, the developers Jiri’s been submitting a number of zine. com/ issue/ 73/ Linux_World_News. intend to proceed as before. If he is not patches for these drivers, so it makes pdf), the question of how to continue released, Alexander’s best guess is that sense he would maintain them if he ReiserFS development came up on the the developers will try to appoint a wished; in any event, no other kernel linux-kernel mailing list. Alexander proxy to run Namesys. hacker has spoken up to claim the role. Lyamin from Hans’s Namesys company offered his take on the situation. He said Status of sysctl Filesystem Benchmarks that ReiserFS 3 has pretty much stabi- In keeping with Linus Torvalds’ recent Some early tests have indicated that ext4 lized into bugfix mode, though Suse assertions that it is never acceptable to is faster with disk writes than either ext3 folks had been adding new features like break user-space, Albert Cahalan volun- or Reiser4. There was general interest in ACL support. So ReiserFS 3 would go on teered to maintain the sysctl code if it these results, though the tests had some as before. couldn’t be removed. But Linus pointed problems (the tester thought delayed In terms of Reiser4, however, Alexan- out that really nothing actually used allocation was part of ext4, when that der said that he and the other Reiser de- sysctl (the implication being that it feature has not yet been merged into velopers were still addressing the techni- wouldn’t actually break anything to get Andrew Morton’s tree).
    [Show full text]
  • A Transparently-Scalable Metadata Service for the Ursa Minor Storage System
    A Transparently-Scalable Metadata Service for the Ursa Minor Storage System Shafeeq Sinnamohideen† Raja R. Sambasivan† James Hendricks†∗ Likun Liu‡ Gregory R. Ganger† †Carnegie Mellon University ‡Tsinghua University Abstract takes to ensure scalability—the visible semantics must be consistent regardless of how the system chooses to The metadata service of the Ursa Minor distributed distribute metadata across servers. Several existing sys- storage system scales metadata throughput as metadata tems have demonstrated this design goal (e.g., [3, 6, 36]). servers are added. While doing so, it correctly han- dles operations that involve metadata served by differ- The requirement to be scalable implies that the sys- ent servers, consistently and atomically updating such tem will use multiple metadata nodes, with each storing metadata. Unlike previous systems, Ursa Minor does so some subset of the metadata and servicing some subset by reusing existing metadata migration functionality to of the metadata requests. In any system with multiple avoid complex distributed transaction protocols. It also servers, it is possible for the load across servers to be un- assigns object IDs to minimize the occurrence of multi- balanced; therefore, some mechanism for load balancing server operations. This approach allows Ursa Minor to is desired. This can be satisfied by migrating some of the implement a desired feature with less complexity than al- metadata from an overloaded server to a less loaded one, ternative methods and with minimal performance penalty thus relocating requests that pertain to that metadata. (under 1% in non-pathological cases). The requirement to be transparent implies that clients 1 Introduction see the same behavior regardless of how the system has distributed metadata across servers.
    [Show full text]
  • Multi-Process Systems: • Information Marshalling and File System Unmarshalling • Block Management • Directory Implementation
    What we have discussed before • The file abstraction • File manager and file descriptors Multi-Process Systems: • Information marshalling and File system unmarshalling • Block management • Directory implementation Operating Systems and Distributed Systems Operating Systems and Distributed Systems 2 What we have discussed before What we have discussed before Fid = open("MyInput.txt", O_RDONLY)) numRead = read(Fid, buf, BUF_LEN)) Operating Systems and Distributed Systems Operating Systems and Distributed Systems The long-term information What we will learn storage problem • Must store large amounts of data • Files – Gigabytes -> terabytes -> petabytes • Directories & naming • Stored information must survive the • File system implementation termination of the process using it • Example file systems – Lifetime can be seconds to years – Must have some way of finding it! • Multiple processes must be able to access the information concurrently Operating Systems and Distributed Systems 5 Operating Systems and6 Distributed Systems What is a file system? The Linux file system • An organization of data and metadata on a storage device • The Linux file system architecture is an interesting example of abstracting complexity. • Another way to think about a file system is as a • Using a common set of API functions, a large variety of file protocol. systems can be supported on a large variety of storage devices. – Just as network protocols (such as IP) give meaning to the streams of data traversing the Internet, file systems • Example: the read function call, which allows some number of give meaning to the data on a particular storage medium. bytes to be read from a given file descriptor. • There are many types of file systems and media.
    [Show full text]
  • Disk Imaging Technologies
    Disk Imaging Technologies Backup and Restoration Challenges Topics • Manufacture Firmware Changes • File System Landscape – UEFI – Introduction to GUID Partition Table (GPT) – Partitions & Limitations • Imaging Utilities Windows & Linux • Full Disk Encryption • Source Web-Links Manufacture Firmware Changes • Industry push to a new standard: – BIOS vs. UEFI • UEFI is to replace and extend the old BIOS firmware. • UEFI is not a new thing. Intel has been working in EFI/UEFI since mid 1990s, and there are vendors like HP or Apple that provided EFI machines since a long time ago. But it is when Microsoft announced Windows 8 that UEFI became the required way to boot the new certified machines. • Secure boot is an extension of UEFI. One of the key points of UEFI is that it can be extended. UEFI has an internal virtual machine that is independent of the architecture that it is using. The standard accepts special binary files compiled for this virtual machine (EFI binaries) that can be executed inside the environment. These binaries can be device drivers, applications or extensions to the UEFI standard. UEFI, in some sense, is like a small operative system that runs when the machine is powered on and whose main task is to find and load another operating system. Unified Extensible Firmware Interface Unified Extensible Firmware Interface (UEFI) is meant as a replacement for the Basic Input/Output System (BIOS) firmware interface ● Initially (1998) designed by Intel for Itanium processor ● Since 2005 managed by the Unified EFI Forum (uefi.org) Source: http://loadays.org/archives/2013/static/slides/Integrating-UEFI-into-rear.pdf Why UEFI? • BIOS has its (aging) limitations – 16-bit processes – max.
    [Show full text]
  • The Evolution of File Systems
    The Evolution of File Systems Thomas Rivera, Hitachi Data Systems Craig Harmer, April 2011 SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. The Evolution of File Systems 2 © 2012 Storage Networking Industry Association. All Rights Reserved. 2 Abstract The File Systems Evolution Over time additional file systems appeared focusing on specialized requirements such as: data sharing, remote file access, distributed file access, parallel files access, HPC, archiving, security, etc. Due to the dramatic growth of unstructured data, files as the basic units for data containers are morphing into file objects, providing more semantics and feature- rich capabilities for content processing This presentation will: Categorize and explain the basic principles of currently available file system architectures (e.g.
    [Show full text]
  • Luca Berton [email protected]
    Linux Day 2006 File System la potenza nasce dall'hard disk Luca Berton [email protected] UNIX File System A UFS volume is composed of the following parts: * a few blocks at the beginning of the partition reserved for boot blocks (which must be initialized separately from the filesystem) * a superblock, containing a magic number identifying this as a UFS filesystem, and some other vital numbers describing this filesystem's geometry and statistics and behavioral tuning parameters * a collection of cylinder groups. Each cylinder group has the following components: o a backup copy of the superblock o a cylinder group header, with statistics, free lists, etc, about this cylinder group, similar to those in the superblock o a number of inodes, each containing file attributes o a number of data blocks Inodes are numbered sequentially. The first several inodes are reserved for historical reasons, followed by the inode for the root directory. Directory files contain only the list of filenames in the directory and the inode associated with each file. All file metadata is kept in the inode. FAT12 FAT16 FAT32 Developer Microsoft Full Name File Allocation Table FAT (12-bit version) (16-bit version) (32-bit version) Introduced 1977 (Microsoft Disk BASIC) July 1988 (MS-DOS 4.0) August 1996 (Windows 95 OSR2) Structures Directory contents Table File allocation Linked List Bad blocks Cluster tagging Limits Max file size 32 MiB 2 GiB 4 GiB Max number of files 4,077 65,517 268,435,437 Max filename size 8.3, or 255 characters when using LFNs Max volume size 32 MiB 2 GiB 4 GiB with some implementations 2 TiB Features Dates recorded Creation, modified, access Date range January 1, 1980 - December 31, 2107 Forks Not natively Attributes Read-only, hidden, system, volume label, subdirectory, archive Permissions No Transparent compression Per-volume, Stacker, DoubleSpace, DriveSpace No Transparent encryption Per-volume only with DR-DOS No problemi - FAT32 ● Scalability: The largest volume size supported by FAT32 on-disk structures is 2 terabytes.
    [Show full text]
  • Extending the Lifetime of Flash-Based Storage Through Reducing Write Amplification from File Systems
    Extending the Lifetime of Flash-based Storage through Reducing Write Amplification from File Systems Youyou Lu Jiwu Shu∗ Weimin Zheng Department of Computer Science and Technology, Tsinghua University Tsinghua National Laboratory for Information Science and Technology ∗Corresponding author: [email protected] [email protected], [email protected] Abstract storage systems. However, the increased density of flash memory requires finer voltage steps inside each cell, Flash memory has gained in popularity as storage which is less tolerant to leakage and noise interference. devices for both enterprise and embedded systems As a result, the reliability and lifetime of flash memory because of its high performance, low energy and reduced have declined dramatically, producing the endurance cost. The endurance problem of flash memory, however, problem [18, 12, 17]. is still a challenge and is getting worse as storage Wear leveling and data reduction are two common density increases with the adoption of multi-level cells methods to extend the lifetime of the flash storage. With (MLC). Prior work has addressed wear leveling and wear leveling, program/erase (P/E) operations tend to be data reduction, but there is significantly less work on distributed across the flash blocks to make them wear out using the file system to improve flash lifetimes. Some evenly [11, 14]. Data reduction is used in both the flash common mechanisms in traditional file systems, such as translation layers (FTLs) and the file systems. The FTLs journaling, metadata synchronization, and page-aligned introduce deduplication and compression techniques to update, can induce extra write operations and aggravate avoid updates of redundant data [15, 33, 34].
    [Show full text]
  • 08-Filesystems.Pdf
    ECE590-03 Enterprise Storage Architecture Fall 2017 File Systems Tyler Bletsch Duke University The file system layer User code open, read, write, seek, close, stat, mkdir, rmdir, unlink, ... Kernel VFS layer File system drivers ext4 fat nfs ... Disk driver NIC driver read_block, write_block packets Could be a single drive or a RAID HDD / SSD 2 High-level motivation • Disks are dumb arrays of blocks. • Need to allocate/deallocate (claim and free blocks) • Need logical containers for blocks (allow delete, append, etc. on different buckets of data – files) • Need to organize such containers (how to find data – directories) • May want to access control (restrict data access per user) • May want to dangle additional features on top Result: file systems 3 Disk file systems • All have same goal: • Fulfill file system calls (open, seek, read, write, close, mkdir, etc.) • Store resulting data on a block device • The big (non-academic) file systems • FAT (“File Allocation Table”): Primitive Microsoft filesystem for use on floppy disks and later adapted to hard drives • FAT32 (1996) still in use (default file system for USB sticks, SD cards, etc.) • Bad performance, poor recoverability on crash, but near-universal and easy for simple systems to implement • ext2, ext3, ext4: Popular Linux file system. • Ext2 (1993) has inode-based on-disk layout – much better scalability than FAT • Ext3 (2001) adds journaling – much better recoverability than FAT • Ext4 (2008) adds various smaller benefits • NTFS: Current Microsoft filesystem (1993). • Like ext3, adds journaling to provide better recoverability than FAT • More expressive metadata (e.g. Access Control Lists (ACLs)) • HFS+: Current Mac filesystem (1998).
    [Show full text]
  • EXPLODE: a Lightweight, General System for Finding Serious Storage System Errors
    EXPLODE: a Lightweight, General System for Finding Serious Storage System Errors Junfeng Yang, Can Sar, and Dawson Engler Computer Systems Laboratory Stanford University Abstract dynamic storage checkers. EXPLODE makes it easy for Storage systems such as file systems, databases, and RAID sys- checkers to find bugs in crash recovery code: as they run tems have a simple, basic contract: you give them data, they do on a live system they tell EXPLODE when to generate the not lose or corrupt it. Often they store the only copy, making disk images that could occur if the system crashed at the its irrevocable loss almost arbitrarily bad. Unfortunately, their current execution point, which they then check for errors. code is exceptionally hard to get right, since it must correctly We explicitly designed EXPLODE so that clients can recover from any crash at any program point, no matter how check complex storage stacks built from many different their state was smeared across volatile and persistent memory. subsystems. For example, Figure 1 shows a version con- This paper describes EXPLODE, a system that makes it trol system on top of NFS on top of the JFS file sys- easy to systematically check real storage systems for errors. It takes user-written, potentially system-specific checkers and tem on top of RAID. EXPLODE makes it quick to assem- uses them to drive a storage system into tricky corner cases, ble checkers for such deep stacks by providing interfaces including crash recovery errors. EXPLODE uses a novel adap- that let users write small checker components and then tation of ideas from model checking, a comprehensive, heavy- plug them together to build many different checkers.
    [Show full text]
  • 25 Example File Systems
    25 Example File Systems Special Features of Files Example File Systems February 9 2009 Winter Term 2008/09 Gerd Liefländer © 2009 Universität Karlsruhe (TH), System Architecture Group 1 Introduction Recommended Reading Bacon, J.: Concurrent Systems (5) Nehmer, J.: Systemsoftware: Grlg. mod. BS, (9) Silberschatz, A.: Operating System Concepts (10,11) Stallings, W.: Operating Systems (12) Tanenbaum, A.: Modern Operating Systems (5, 6) © 2009 Universität Karlsruhe (TH), System Architecture Group 2 Introduction Roadmap for Today Special Features of Files & File Systems File Control Structures Memory Mapped Files Log Structured FS Example File Systems Unix BSD FFS EXT2 Linux VFS Reiser FS © 2009 Universität Karlsruhe (TH), System Architecture Group 3 File Control Structures © 2009 Universität Karlsruhe (TH), System Architecture Group 4 File Control Structure File Control Block Per application there is a list of opened files Per opened file there is a file control block (FCB) Position pointer Current block address Links to buffers in main memory Filling grade of buffer Lock information Access dates (e.g. time when file was opened) Access rights © 2009 Universität Karlsruhe (TH), System Architecture Group 5 File Control Structure Unix FCBs per Task/Process Per default each task has a couple of standard files stdin FID = 0 stdout FID = 1 stderr FID = 2 FIDs with higher value are used for other files Once a file is closed, its FID is never used again With FIDs it is easy to redirect output to different files and
    [Show full text]