DOCSLIB.ORG

Native LINUX Filesystems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Native LINUX Filesystems Native LINUX Filesystems Extended filesystems (Ext, Ext2, Ext3, Ext4) Extended filesystem (ext fs), second extended filesystem (ext2fs) and third extended filesystem (ext3fs) were designed and implemented on Linux by Rmy Card, Laboratoire MASI--Institut Blaise Pascal, , Theodore Ts'o, Massachussets Institute of Technology, and Stephen Tweedie, University of Edinburgh Non-Journaled Filesystems Extended filesystem (Ext FS) This is original filesystem used in early Linux systems. The standard filesystem for Linux, ext2, is a high-performance, non-journaled filesystem. Although ext2 lacks journaling features, many users choose it because of its high speed and reliability. Second Extended Filesystem (Ext2 FS) The Second Extended File System provides standard Unix file semantics and advanced features. Ext2 filesystem format forms the basis for following native LINUX file system versions. Due to optimizations included in the kernel code, Ext2fs has extensions to the current filesystem: access control lists conforming to the Posix semantics, undelete, and on-the-fly file compression. I Ext2 features: - Long file names (255 characters to 1012_ and variable length directory entries. - VFS layer filesystems to 4 TB - Reserves 5% of the blocks super user (root) to recover from user processes filling up filesystems. - Filesystem metadata (inodes, bitmap blocks, indirect blocks and directory blocks) synchronous write - Choice of logical block size when creating the filesystem, typically be 1024, 2048 and 4096 bytes to speed up I/O since with fewer I/O requests, and thus fewer disk head seeks. - Fast symbolic links that do not use any data block on the filesystem; filename is not stored in the inode - filesystem state using a special field in the superblock to indicate the status of the file system. When a filesystem is mounted in read/write mode, its state is set to ``Not Clean''. When it is unmounted or remounted in read-only mode, its state is reset to ``Clean''. At boot time, the filesystem checker uses this information to decide if a filesystem must be checked (fsck)... The filesystem checker tests this to force the check of the filesystem regardless of its apparently clean state (fsck). - Filesystems checks are forced at regular intervals. A mount counter is maintained in the superblock. A last check time and a maximal check interval are also maintained in the superblock. Each time the filesystem is mounted in read/write mode, counters and timestamps arechecked. When it reaches a maximal value (also recorded in the superblock), the filesystem checker forces the check even if the filesystem is ``Clean''. - provides an attribute allows the users to request secure deletion on files. When such a file is deleted, random data is written in the disk blocks previously allocated to the file. This prevents malicious people from gaining access to the previous content of the file by using a disk editor. Ext2 Physical Structure Unlike FFS, the ext2 filesystems is made up of block groups instead of FFS cylinder groups. Block groups are not tied to the physical layout of the blocks on the disk, since modern drives tend to be optimized for sequential access (‘smart” drives – SAN, SCSI, SATA) and hide their physical geometry to the operating system. Ext2 filesystem layout ,---------+---------+---------+---------+---------, | Boot | Block | Block | ... | Block | | sector | group 1 | group 2 | | group n | `---------+---------+---------+---------+---------' Each block group contains a redundant copy of crucial filesystem control informations (superblock and the filesystem descriptors) and also contains a part of the filesystem (a block bitmap, an inode bitmap, a piece of the inode table, and data blocks). The structure of a block group is represented in this table: Ext2 blockgroup layout ,---------+---------+---------+---------+---------+---------, | Super | FS | Block | Inode | Inode | Data | | block | desc. | bitmap | bitmap | table | blocks | ---------+---------+---------+---------+---------+---------' Using block groups improves reliability since the control structures are replicated in each block group, it is easy to recover from a filesystem where the superblock has been corrupted. This structure also helps to get good performances: by reducing the distance between the inode table and the data blocks, it is possible to reduce the disk head seeks during I/O on files. - In Ext2fs, directories are managed as linked lists of variable length entries containing the inode number, the entry length, the file name and its length. Variable length entries permit long file names without wasting disk space in directories. - Ext2fs buffer cache management performs readaheads:reading data blocks contiguouslys. This way, it tries to ensure that the next block to read will already be loaded into the buffer cache. Readaheads are extended directory reads - Ext2fs performs allocation optimizations. Block groups are used to cluster together related inodes and data to reduce the disk head seeks made when the kernel reads an inode and its data blocks. - Preallocates up to 8 adjacent blocks when allocating a new block for writing data. Preallocation hit rates are around 75% even on very full filesystems and gets good write performances under heavy load. It also allows contiguous blocks to be allocated to files, thus it speeds up the future sequential reads. Journaled Filesystems Journaled filesystems include additional record keeping that increases the ability of the filesystem to recover from a crash. · ext3 - the ext2 filesystem with journaling extensions. · jfs - Journaled File System - a filesystem contributed to Linux by IBM. · xfs - A filesystem contributed to open source by SGI. · reiserfs, developed by Namesys, is the default filesystem for SUSE Linux, DARPA. •Third Extended Filesystem (Ext3 FS) Ext3 supports the same features as Ext2, but also includes Journaling. • Fourth Extended Filesystem (Ext4 FS) Compatibility Any existing Ext3 filesystem can be migrated to Ext4 with an easy procedure which consists in running a couple of commands in read-only mode. Migrate existing Ext3 filesystems to Ext4 You need to use the tune2fs and fsck tools in the filesystem, and that filesystem needs to be unmounted. Run: tune2fs -O extents,uninit_bg,dir_index /dev/yourfilesystem After running this command you MUST run fsck. If you don't do it, Ext4 WILL NOT MOUNT your filesystem. This fsck run is needed to return the filesystem to a consistent state. It WILL tell you that it finds checksum errors in the group descriptors - it's expected, and it's exactly what it needs to be rebuilt to be able to mount it as Ext4, so don't get surprised by them. Since each time it finds one of those errors it asks you what to do, always say YES. If you don't want to be asked, add the "-p" parameter to the fsck command, it means "automatic repair": (e2)fsck -pfDCO /dev/yourfilesystem Bigger filesystem/file sizes Currently, Ext3 support 16 TB of maximum filesystem size, and 2 TB of maximum file size. Ext4 adds 48-bit block addressing, so it will have 1 EB of maximum filesystem size and 16 TB of maximum file size. 1 EB = 1,048,576 TB (1 EB = 1024 PB, 1 PB = 1024 TB, 1 TB = 1024 GB). Sub directory scalability Right now the maximum possible number of sub directories contained in a single directory in Ext3 is 32000. Ext4 breaks that limit and allows a unlimited number of sub directories. Extents The traditionally Unix-derived filesystems like Ext3 use a indirect block mapping scheme to keep track of each block used for the blocks corresponding to the data of a file. This is inefficient for large files, especially on large file delete and truncate operations, because the mapping keeps a entry for every single block, and big files have many blocks -> huge mappings, slow to handle. Modern filesystems use a different approach called "extents". An extent is basically a bunch of contiguous physical blocks. Multiblock allocation When Ext3 needs to write new data to the disk, there's a block allocator that decides which free blocks will be used to write the data. But the Ext3 block allocator only allocates one block (4KB) at a time. That means that if the system needs to write the 100 MB data mentioned in the previous point, it will need to call the block allocator 25600 times (and it was just 100 MB!). Ext4 uses a "multiblock allocator" (mballoc) which allocates many blocks in a single call, instead of a single block per call, avoiding a lot of overhead. This improves the performance, and it's particularly useful with delayed allocation and extents. This feature doesn't affect the disk format. Also, note that the Ext4 block/inode allocator has other improvements, described in detail in this paper. Delayed allocation Delayed allocation is a performance feature (it doesn't change the disk format) found in a few modern filesystems such as XFS, ZFS, btrfs or Reiser 4, and it consists in delaying the allocation of blocks as much as possible, contrary to what traditionally filesystems (such as Ext3, reiser3, etc) do: allocate the blocks as soon as possible. EXT4 Delayed allocation, on the other hand, does not allocate the blocks immediately when the process write()s, rather, it delays the allocation of the blocks while the file is kept in cache, until it is really going to be written to the disk. This gives the block allocator the opportunity to optimize the allocation in situations where the old system couldn't. Fast fsck Fsck is a very slow operation, especially the first step: checking all the inodes in the file system. In Ext4, at the end of each group's inode table will be stored a list of unused inodes (with a checksum, for safety), so fsck will not check those inodes. The result is that total fsck time improves from 2 to 20 times, depending on the number of used inodes. Journal checksumming The journal is the most used part of the disk, making the blocks that form part of it more prone to hardware failure.
Load more

Recommended publications
  • CS 5600 Computer Systems
    CS 5600 Computer Systems Lecture 10: File Systems What are We Doing Today? • Last week we talked extensively about hard drives and SSDs – How they work – Performance characterisEcs • This week is all about managing storage – Disks/SSDs offer a blank slate of empty blocks – How do we store files on these devices, and keep track of them? – How do we maintain high performance? – How do we maintain consistency in the face of random crashes? 2 • ParEEons and MounEng • Basics (FAT) • inodes and Blocks (ext) • Block Groups (ext2) • Journaling (ext3) • Extents and B-Trees (ext4) • Log-based File Systems 3 Building the Root File System • One of the first tasks of an OS during bootup is to build the root file system 1. Locate all bootable media – Internal and external hard disks – SSDs – Floppy disks, CDs, DVDs, USB scks 2. Locate all the parEEons on each media – Read MBR(s), extended parEEon tables, etc. 3. Mount one or more parEEons – Makes the file system(s) available for access 4 The Master Boot Record Address Size Descripon Hex Dec. (Bytes) Includes the starEng 0x000 0 Bootstrap code area 446 LBA and length of 0x1BE 446 ParEEon Entry #1 16 the parEEon 0x1CE 462 ParEEon Entry #2 16 0x1DE 478 ParEEon Entry #3 16 0x1EE 494 ParEEon Entry #4 16 0x1FE 510 Magic Number 2 Total: 512 ParEEon 1 ParEEon 2 ParEEon 3 ParEEon 4 MBR (ext3) (swap) (NTFS) (FAT32) Disk 1 ParEEon 1 MBR (NTFS) 5 Disk 2 Extended ParEEons • In some cases, you may want >4 parEEons • Modern OSes support extended parEEons Logical Logical ParEEon 1 ParEEon 2 Ext.
    [Show full text]
  • Ext4 File System and Crash Consistency
    1 Ext4 file system and crash consistency Changwoo Min 2 Summary of last lectures • Tools: building, exploring, and debugging Linux kernel • Core kernel infrastructure • Process management & scheduling • Interrupt & interrupt handler • Kernel synchronization • Memory management • Virtual file system • Page cache and page fault 3 Today: ext4 file system and crash consistency • File system in Linux kernel • Design considerations of a file system • History of file system • On-disk structure of Ext4 • File operations • Crash consistency 4 File system in Linux kernel User space application (ex: cp) User-space Syscalls: open, read, write, etc. Kernel-space VFS: Virtual File System Filesystems ext4 FAT32 JFFS2 Block layer Hardware Embedded Hard disk USB drive flash 5 What is a file system fundamentally? int main(int argc, char *argv[]) { int fd; char buffer[4096]; struct stat_buf; DIR *dir; struct dirent *entry; /* 1. Path name -> inode mapping */ fd = open("/home/lkp/hello.c" , O_RDONLY); /* 2. File offset -> disk block address mapping */ pread(fd, buffer, sizeof(buffer), 0); /* 3. File meta data operation */ fstat(fd, &stat_buf); printf("file size = %d\n", stat_buf.st_size); /* 4. Directory operation */ dir = opendir("/home"); entry = readdir(dir); printf("dir = %s\n", entry->d_name); return 0; } 6 Why do we care EXT4 file system? • Most widely-deployed file system • Default file system of major Linux distributions • File system used in Google data center • Default file system of Android kernel • Follows the traditional file system design 7 History of file system design 8 UFS (Unix File System) • The original UNIX file system • Design by Dennis Ritche and Ken Thompson (1974) • The first Linux file system (ext) and Minix FS has a similar layout 9 UFS (Unix File System) • Performance problem of UFS (and the first Linux file system) • Especially, long seek time between an inode and data block 10 FFS (Fast File System) • The file system of BSD UNIX • Designed by Marshall Kirk McKusick, et al.
    [Show full text]
  • W4118: Linux File Systems
    W4118: Linux file systems Instructor: Junfeng Yang References: Modern Operating Systems (3rd edition), Operating Systems Concepts (8th edition), previous W4118, and OS at MIT, Stanford, and UWisc File systems in Linux Linux Second Extended File System (Ext2) . What is the EXT2 on-disk layout? . What is the EXT2 directory structure? Linux Third Extended File System (Ext3) . What is the file system consistency problem? . How to solve the consistency problem using journaling? Virtual File System (VFS) . What is VFS? . What are the key data structures of Linux VFS? 1 Ext2 “Standard” Linux File System . Was the most commonly used before ext3 came out Uses FFS like layout . Each FS is composed of identical block groups . Allocation is designed to improve locality inodes contain pointers (32 bits) to blocks . Direct, Indirect, Double Indirect, Triple Indirect . Maximum file size: 4.1TB (4K Blocks) . Maximum file system size: 16TB (4K Blocks) On-disk structures defined in include/linux/ext2_fs.h 2 Ext2 Disk Layout Files in the same directory are stored in the same block group Files in different directories are spread among the block groups Picture from Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 3 Block Addressing in Ext2 Twelve “direct” blocks Data Data BlockData Inode Block Block BLKSIZE/4 Indirect Data Data Blocks BlockData Block Data (BLKSIZE/4)2 Indirect Block Data BlockData Blocks Block Double Block Indirect Indirect Blocks Data Data Data (BLKSIZE/4)3 BlockData Data Indirect Block BlockData Block Block Triple Double Blocks Block Indirect Indirect Data Indirect Data BlockData Blocks Block Block Picture from Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc.
    [Show full text]
  • Ext3 = Ext2 + Journaling
    FS Sistem datoteka-skup metoda i struktura podataka koje operativni sistem koristi za čuvanje podataka Struktura sistema datoteka: - 1. zaglavlje→neophodni podaci za funkcionisanje sistema datoteka - 2. strukture za organizaciju podataka na medijumu→meta podaci - 3. podaci→datoteke i direktorijumi Strukture podataka neophodne za realizaciju sistema datoteka: - PCB(Partition Control Block) - BCB(Boot control Block) - Kontrolne strukture za alokaciju datoteka(i-node tabela kod Linux-a) - Direktorijumske strukture koje sadrže kontrolne blokove datoteka - FCB(File Control Block) ext3 Slide 1 of 51 VIRTUELNI SISTEM DATOTEKA(VFS) Linux podržava rad sa velikim brojem sistema datoteka(ext2,ext3, XFS,FAT, NTFS...) VFS-objektno orjentisani način realizacije sistema datoteka koji omogućava korisniku da na isti način pristupa svim sistemima datoteka Način obraćanja korisnika sistemu datoteka - korisnik->API - VFS->sistem datoteka ext3 Slide 2 of 51 Linux FS Linux posmatra svaki sistem datoteka kao nezavisnu hijerarhijsku strukturu objekata(datoteka i direktorijuma) na čijem se vrhu nalazi root(/) direktorijum Objekti Linux sistema datoteka: Super block - zaglavlje(superblock) - i-node tabela I-Node Table - blokovi sa podacima - direktorijumski blokovi - blokovi indirektnih pokazivača Data Area i-node-opisuje objekte, oko 128B na disku Kompromis između veličine i-node tabele i brzine rada sistema datoteka - prvih 10-12 pokazivača na blokove sa podacima - za alokaciju većih datoteka koristi se single indirection block - za još veće datoteke
    [Show full text]
  • Comparing Filesystem Performance: Red Hat Enterprise Linux 6 Vs
    COMPARING FILE SYSTEM I/O PERFORMANCE: RED HAT ENTERPRISE LINUX 6 VS. MICROSOFT WINDOWS SERVER 2012 When choosing an operating system platform for your servers, you should know what I/O performance to expect from the operating system and file systems you select. In the Principled Technologies labs, using the IOzone file system benchmark, we compared the I/O performance of two operating systems and file system pairs, Red Hat Enterprise Linux 6 with ext4 and XFS file systems, and Microsoft Windows Server 2012 with NTFS and ReFS file systems. Our testing compared out-of-the-box configurations for each operating system, as well as tuned configurations optimized for better performance, to demonstrate how a few simple adjustments can elevate I/O performance of a file system. We found that file systems available with Red Hat Enterprise Linux 6 delivered better I/O performance than those shipped with Windows Server 2012, in both out-of- the-box and optimized configurations. With I/O performance playing such a critical role in most business applications, selecting the right file system and operating system combination is critical to help you achieve your hardware’s maximum potential. APRIL 2013 A PRINCIPLED TECHNOLOGIES TEST REPORT Commissioned by Red Hat, Inc. About file system and platform configurations While you can use IOzone to gauge disk performance, we concentrated on the file system performance of two operating systems (OSs): Red Hat Enterprise Linux 6, where we examined the ext4 and XFS file systems, and Microsoft Windows Server 2012 Datacenter Edition, where we examined NTFS and ReFS file systems.
    [Show full text]
  • NOVA: a Log-Structured File System for Hybrid Volatile/Non
    NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories Jian Xu and Steven Swanson, University of California, San Diego https://www.usenix.org/conference/fast16/technical-sessions/presentation/xu This paper is included in the Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST ’16). February 22–25, 2016 • Santa Clara, CA, USA ISBN 978-1-931971-28-7 Open access to the Proceedings of the 14th USENIX Conference on File and Storage Technologies is sponsored by USENIX NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories Jian Xu Steven Swanson University of California, San Diego Abstract Hybrid DRAM/NVMM storage systems present a host of opportunities and challenges for system designers. These sys- Fast non-volatile memories (NVMs) will soon appear on tems need to minimize software overhead if they are to fully the processor memory bus alongside DRAM. The result- exploit NVMM’s high performance and efficiently support ing hybrid memory systems will provide software with sub- more flexible access patterns, and at the same time they must microsecond, high-bandwidth access to persistent data, but provide the strong consistency guarantees that applications managing, accessing, and maintaining consistency for data require and respect the limitations of emerging memories stored in NVM raises a host of challenges. Existing file sys- (e.g., limited program cycles). tems built for spinning or solid-state disks introduce software Conventional file systems are not suitable for hybrid mem- overheads that would obscure the performance that NVMs ory systems because they are built for the performance char- should provide, but proposed file systems for NVMs either in- acteristics of disks (spinning or solid state) and rely on disks’ cur similar overheads or fail to provide the strong consistency consistency guarantees (e.g., that sector updates are atomic) guarantees that applications require.
    [Show full text]
  • Measuring Parameters of the Ext4 File System
    File System Forensics : Measuring Parameters of the ext4 File System Madhu Ramanathan Venkatesh Karthik Srinivasan Department of Computer Sciences, UW Madison Department of Computer Sciences, UW Madison [email protected] [email protected] Abstract An extent is a group of physically contiguous blocks. Allocating Operating systems are rather complex software systems. The File extents instead of indirect blocks reduces the size of the block map, System component of Operating Systems is defined by a set of pa- thus, aiding the quick retrieval of logical disk block numbers and rameters that impact both the correct functioning as well as the per- also minimizes external fragmentation. An extent is represented in formance of the File System. In order to completely understand and an inode by 96 bits with 48 bits to represent the physical block modify the behavior of the File System, correct measurement of number and 15 bits to represent length. This allows one extent to have a length of 215 blocks. An inode can have at most 4 extents. those parameters and a thorough analysis of the results is manda- 15 tory. In this project, we measure the various key parameters and If the file is fragmented, every extent typically has less than 2 a few interesting properties of the Fourth Extended File System blocks. If the file needs more than four extents, either due to frag- (ext4). The ext4 has become the de facto File System of Linux ker- mentation or due to growth, an extent HTree rooted at the inode is nels 2.6.28 and above and has become the default file system of created.
    [Show full text]
  • Migrating from Netware to OES 2 Linux
    Best Practice Guide www.novell.com Migrating from NetWare to OES 2 prepared for Novell OES 2 User Community Published: November, 2007 Disclaimer Novell, Inc. makes no representations or warranties with respect to the contents or use of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Trademarks Novell is a registered trademark of Novell, Inc. in the United States and other countries. * All third-party trademarks are property of their respective owner. Copyright 2007 Novell, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of Novell, Inc. Novell, Inc. 404 Wyman Suite 500 Waltham Massachusetts 02451 USA Prepared By Novell Services and User Community Migrating from NetWare to OES 2—Best Practice Guide November, 2007 Novell OES 2 User Community The latest version of this document, along with other OES 2 Linux Best Practice Guides, can be found with the NetWare to Linux Migration Resources at: http://www.novell.com/products/openenterpriseserver/netwaretolinux/view/all/-9/tle/all Contents Acknowledgments.................................................................................. iv Getting Started...................................................................................... 1 Why OES 2?..............................................................................................1 Which Services Are Right for OES 2? ................................................................4
    [Show full text]
  • Filesystem Considerations for Embedded Devices ELC2015 03/25/15
    Filesystem considerations for embedded devices ELC2015 03/25/15 Tristan Lelong Senior embedded software engineer Filesystem considerations ABSTRACT The goal of this presentation is to answer a question asked by several customers: which filesystem should you use within your embedded design’s eMMC/SDCard? These storage devices use a standard block interface, compatible with traditional filesystems, but constraints are not those of desktop PC environments. EXT2/3/4, BTRFS, F2FS are the first of many solutions which come to mind, but how do they all compare? Typical queries include performance, longevity, tools availability, support, and power loss robustness. This presentation will not dive into implementation details but will instead summarize provided answers with the help of various figures and meaningful test results. 2 TABLE OF CONTENTS 1. Introduction 2. Block devices 3. Available filesystems 4. Performances 5. Tools 6. Reliability 7. Conclusion Filesystem considerations ABOUT THE AUTHOR • Tristan Lelong • Embedded software engineer @ Adeneo Embedded • French, living in the Pacific northwest • Embedded software, free software, and Linux kernel enthusiast. 4 Introduction Filesystem considerations Introduction INTRODUCTION More and more embedded designs rely on smart memory chips rather than bare NAND or NOR. This presentation will start by describing: • Some context to help understand the differences between NAND and MMC • Some typical requirements found in embedded devices designs • Potential filesystems to use on MMC devices 6 Filesystem considerations Introduction INTRODUCTION Focus will then move to block filesystems. How they are supported, what feature do they advertise. To help understand how they compare, we will present some benchmarks and comparisons regarding: • Tools • Reliability • Performances 7 Block devices Filesystem considerations Block devices MMC, EMMC, SD CARD Vocabulary: • MMC: MultiMediaCard is a memory card unveiled in 1997 by SanDisk and Siemens based on NAND flash memory.
    [Show full text]
  • Digital Forensics Focus Area
    Digital Forensics Focus Area Barbara Guttman Forensics@NIST November 6, 2020 Digital Forensics – Enormous Scale • Computer crime is now a big volume crime (even worse now that everyone is online) - both in the number of cases and the impacts of the crimes. • Estimate of 6000% increase in spam (claiming to be PPP, WHO) • Most serious crimes have a nexus to digital forensics: Drug dealing uses phones and drones. Murderers communicate with their victims. Financial fraud uses computers. Child sexual exploitation is recorded and shared online. • 2018: Facebook sent 45 million images to LE • Digital Forensics used to support investigations and prosecutions • Used by Forensics Labs, Lawyers, Police Digital Forensics Overview Digital Forensics goal: Provide trustworthy, useful and timely information Digital Forensics has several problems meeting this goal: 1. Overwhelmed with the volume of material 2. Overwhelmed with the variety of material, the constant change and the technical skills needed to understand the material Digital Forensics needs: 1. High quality tools and techniques 2. Help with operational quality and efficiency Digital Forensic Projects • National Software Reference Library • Computer Forensics Tool Testing • Federated Testing • Computer Forensics Reference Dataset • Tool Catalog • Black Box Study and Digital Forensics Scientific Foundation National Software Reference Library (NSRL) The National Software Reference Library (NSRL): • Collects software • Populates a database with software metadata, individual files, file hashes •
    [Show full text]
  • Outline of Ext4 File System & Ext4 Online Defragmentation Foresight
    Outline of Ext4 File System & Ext4 Online Defragmentation Foresight LinuxCon Japan/Tokyo 2010 September 28, 2010 Akira Fujita <[email protected]> NEC Software Tohoku, Ltd. Self Introduction ▐ Name: Akira Fujita Japan ▐ Company: NEC Software Tohoku, Ltd. in Sendai, Japan. Sendai ● ▐ Since 2004, I have been working at NEC Software Tohoku developing Linux file system, mainly ext3 and ● ext4 filesystems. Tokyo Currently, I work on the quality evaluation of ext4 for enterprise use, and also develop the ext4 online defragmentation. Page 2 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. Outline ▐ What is ext4 ▐ Ext4 features ▐ Compatibility ▐ Performance measurement ▐ Recent ext4 topics ▐ What is ext4 online defrag ▐ Relevant file defragmentation ▐ Current status / future plan Page 3 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. What is ext4 ▐ Ext4 is the successor of ext3 which is developed to solve performance issues and scalability bottleneck on ext3 and also provide backward compatibility with ext3. ▐ Ext4 development began in 2006. Included in stable kernel 2.6.19 as EXPERIMENTAL (ext4dev). Since kernel 2.6.28, ext4 has been released as stable (Renamed from ext4dev to ext4 in kernel 2.6.28). ▐ Maintainers Theodore Ts'o [email protected] , Andreas Dilger [email protected] ▐ ML [email protected] ▐ Ext4 Wiki http://ext4.wiki.kernel.org Page 4 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. Ext4 features Page 5 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. Ext4 features Bigger file/filesystem size support. Compared to ext3, ext4 is: 8 times larger in file size, 65536 times(!) larger in filesystem size.
    [Show full text]
  • User Manual for Data Backups
    1 User Manual for Data Backups Accepted formats are: EXT3, EXT4, NTFS, XFS, FAT32, exFAT and HFS+ (Mac OS). Recommended format: EXT3 and EXT4 Keep in mind that the HFS+ format has limited support on Linux; the drivers change frequently and even the latest drivers may fail. Therefore, if possible, AVOID backing up on Mac OS formatted disks. If the HFS+ is still used, journaling on the disk may need to be disabled first, especially if you plan to Connect the drive to a workstation other than 4 or 7. Details on journal disabling can be found here: https://www.gmca.aps.anl.gov/userprogram/hfsplus.html USB3 drives are the only supported choice because they are faster than USB2, eSATA or Firewire. A USB3 connector can be recognized by its blue-colored socket or by its logo: DO NOT USE ANY SPACES OR SPECIAL CHARACTERS IN THE FILE OR DIRECTORY NAMES! Mounting backup disks and preparing for backups 1. Log in with your account on the Linux workstation. 2. If needed, plug the backup disk in the nearby outlet or the power strip (provided by the GM/CA staff). 3. Connect the disk to the same computer with USB3 data cable. 4. Power up the disk if needed - some disks have power switch and some don't. A window should pop up with the disk name (Fig. 1A). Full path to the backup disk is /run/media/username/DiskLabel, or /run/media/user0/NS4 for the disk on Fig. 1. At the same time, an icon with the same name will appear on the desktop (Fig 1B).
    [Show full text]