DOCSLIB.ORG

File Systems: Semantics & Structure What Is a File

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
File Systems: Semantics & Structure What Is a File 5/15/2017 File Systems: Semantics & Structure What is a File 11A. File Semantics • a file is a named collection of information 11B. Namespace Semantics • primary roles of file system: 11C. File Representation – to store and retrieve data – to manage the media/space where data is stored 11D. Free Space Representation • typical operations: 11E. Namespace Representation – where is the first block of this file 11L. Disk Partitioning – where is the next block of this file 11F. File System Integration – where is block 35 of this file – allocate a new block to the end of this file – free all blocks associated with this file File Systems Semantics and Structure 1 File Systems Semantics and Structure 2 Data and Metadata Sequential Byte Stream Access • File systems deal with two kinds of information int infd = open(“abc”, O_RDONLY); int outfd = open(“xyz”, O_WRONLY+O_CREATE, 0666); • Data – the contents of the file if (infd >= 0 && outfd >= 0) { – e.g. instructions of the program, words in the letter int count = read(infd, buf, sizeof buf); Metadata – Information about the file • while( count > 0 ) { e.g. how many bytes are there, when was it created – write(outfd, buf, count); sometimes called attributes – count = read(infd, inbuf, BUFSIZE); • both must be persisted and protected } – stored and connected by the file system close(infd); close(outfd); } File Systems Semantics and Structure 3 File Systems Semantics and Structure 4 Random Access Consistency Model void *readSection(int fd, struct hdr *index, int section) { struct hdr *head = &hdr[section]; • When do new readers see results of a write? off_t offset = head->section_offset; – read-after-write size_t len = head->section_length; • as soon as possible, data-base semantics void *buf = malloc(len); if (buf != NULL) { • this commonly called “POSIX consistency” lseek(fd, offset, SEEK_SET); – read-after-close (or sync/commit) if ( read(fd, buf, len) <= 0) { free(buf); • only after writes are committed to storage buf = NULL; – open-after-close (or sync/commit) } • each open sees a consistent snapshot } return(buf); – explicitly versioned files } • each open sees a named, consistent snapshot File Systems Semantics and Structure 5 File Systems Semantics and Structure 6 1 5/15/2017 File Attributes – basic properties Extended File Types and Attributes • thus far we have focused on a simple model • extended protection information – a file is a "named collection of data blocks" – e.g. access control lists • in most OS files have more state than this • resource forks – file type (regular file, directory, device, IPC port, ...) – e.g. configuration data, fonts, related objects – file length (may be excess space at end of last block)) • application defined types – ownership and protection information – e.g. load modules, HTML, e-mail, MPEG, ... – system attributes (e.g. hidden, archive) – creation time, modification time, last accessed time • application defined properties • typically stored in file descriptor structure – e.g. compression scheme, encryption algorithm, ... File Systems Semantics and Structure 7 File Systems Semantics and Structure 8 Databases Object Stores simplified file systems, cloud storage • a tool managing business critical data • – optimized for large but infrequent transfers • table is equivalent of a file system • bucket is equivalent of a file system • data organized in rows and columns – a bucket contains named, versioned objects – row indexed by unique key • objects have long names in a flat name space – columns are named fields within each row – object names are unique within a bucket • support a rich set of operations • an object is a blob of immutable bytes – multi-object, read/modify/write transactions – get … all or part of the object – SQL searches return consistent snapshots – put … new version, there is no append/update – insert/delete row/column operations – delete File Systems Semantics and Structure 9 File Systems Semantics and Structure 10 Key-Value Stores File Names and Name Binding • smaller and faster than an SQL database • file system knows files by internal descriptors – optimized for frequent small transfers • users know files by names • table is equivalent of a file system – names more easily remembered than disk addresses – a table is a collection of key/value pairs – names can be structured to organize millions of files • keys have long names in a flat name space • file system responsible for name-to-file mapping – associating names with new files – key names are unique within a table – changing names associated with existing files value is a (typically 64-64MB) string • – allowing users to search the name space – get/put (entire value) • there are many ways to structure a name space – delete File Systems Semantics and Structure 11 File Systems Semantics and Structure 12 2 5/15/2017 What is in a Name? Flat Name Spaces directory • there is one naming context per file system /home /mark /TODO. txt – all file names must be unique within that context suffix separator base name • all files have exactly one true name – these names are probably very long • suffixes and file types • file names may have some structure – file-to-application binding often based on suffix e.g. CAC101.CS111.SECTION1.SLIDES.LECTURE_13 • defined by system configuration registry – • configured per user, or per directory – this structure may be used to optimize searches – suffix may define the file type (e.g. Windows) – the structure is very useful to users – suffix may only be a hint (magic # defines type) – the structure has no meaning to the file system File Systems Semantics and Structure 13 File Systems Semantics and Structure 14 Hierarchical Namespaces A rooted directory tree • directory root – a file containing references to other files – it can be used as a naming context user_1 user_2 user_3 • each process has a current working directory • names are interpreted relative to directory file_a dir_a file_b file_c dir_a • nested directories can form a tree (/user_1/file_a) (/user_1/dir_a) (/user_2/file_b) (/user_3/file_c) (/user_3/dir_a) – file name is a path through that tree – directory tree expands from a root node file_a file_b (/user_1/dir_a/file_a) (/user_3/dir_a/file_b) • fully qualified names begin from the root – may actually form a directed graph File Systems Semantics and Structure 15 File Systems Semantics and Structure 16 True Names vs. Path Names Hard Links: example • Some file systems have “true names” root • DOS and ISO9660 have a single “path name” user_1 user_3 – files are described by directory entries – data is referred to by exactly one directory entry dir_a file_c – each file has only one (character string) name file_a ln /user_3/dir_a/file_b /user_1/file_a • Unix (and Linux) … have named links file_b – files are described by I-nodes (w/unique I#) – directories associate names with I-node numbers – many directory entries can refer to same I-node Both names now refer to the same I-node File Systems Semantics and Structure 17 File Systems Semantics and Structure 18 3 5/15/2017 Unix-style Hard Links Symbolic Links: example • all protection information is stored in the file root ln –s /user_3/dir_a/file_b /user_1/file_a – file owner sets file protection (e.g. read-only) user_1 user_3 – all links provide the same access to the file – anyone with read access to file can create new link file_a dir_a file_c – but directories are protected files too • not everyone has read or search access to every directory /user_3/dir_a/file_b file_b • all links are equal – there is nothing special about the owner‘s link – file is not deleted until no links remain to file – reference count keeps track of references /user_1/file_a is now a macro for /user_3/dir_a/file_b File Systems Semantics and Structure 19 File Systems Semantics and Structure 20 Symbolic Links Generalized Directories: Issues • another type of special file • Can there be multiple links to a single file? – an indirect reference to some other file – who can create new references to a file? – contents is a path name to another file – if one reference is deleted, does the file go away? • Operating System recognizes symbolic links – can the file's owner cancel other peoples' access? – automatically opens associated file instead • Can clients work their way back up the tree? – if file is inaccessible or non-existent, the open fails • Is the namespace truly a tree • symbolic link is not a reference to the I-node – or is it an a-cyclic directed graph – symbolic links will not prevent deletion – or an arbitrary directed graph – do not guarantee ability to follow the specified path – Internet URLs are similar to symbolic links • Does namespace span multiple file systems? File Systems Semantics and Structure 21 File Systems Semantics and Structure 22 File System Goals File System Structure • ensure the privacy and integrity of all files • disk volumes are divided into fixed-sized blocks • efficiently implement name-to-file binding – many sizes are used: 512, 1024, 2048, 4096, 8192 ... – find file associated with this name • most of them will store user data – list the file names in this part of the name space • some will store organizing “meta-data” • efficiently manage data associated w/each file – description of the file system (e.g. layout and state) – file control blocks to describe individual files – return data at offset X in file Y – lists of free blocks (not yet allocated to any file) – write data Z at offset X in file Y • all operating systems have such data structures • manage attributes associated w/each file – different OS and FS often have very different goals – what is the length of file Y – these result in very different implementations – change owner/protection of file Y to be X File Systems Semantics and Structure 23 File Systems Semantics and Structure 24 4 5/15/2017 Unix System 5 – Volume Structure File Descriptor Structures block 0 boot block • all file systems have file descriptor structures super block size and number of I-nodes are block 1 • contain all info about file UNIX I-node block specified in super block – type (e.g.
Load more

Recommended publications
  • Technical Standard
    Technical Standard X/Open Curses, Issue 7 The Open Group ©November 2009, The Open Group All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owners. Technical Standard X/Open Curses, Issue 7 ISBN: 1-931624-83-6 Document Number: C094 Published in the U.K. by The Open Group, November 2009. This standardhas been prepared by The Open Group Base Working Group. Feedback relating to the material contained within this standardmay be submitted by using the web site at http://austingroupbugs.net with the Project field set to "Xcurses Issue 7". ii Technical Standard 2009 Contents Chapter 1 Introduction........................................................................................... 1 1.1 This Document ........................................................................................ 1 1.1.1 Relationship to Previous Issues ......................................................... 1 1.1.2 Features Introduced in Issue 7 ........................................................... 2 1.1.3 Features Withdrawn in Issue 7........................................................... 2 1.1.4 Features Introduced in Issue 4 ........................................................... 2 1.2 Conformance............................................................................................ 3 1.2.1 Base Curses Conformance .................................................................
    [Show full text]
  • Allgemeines Abkürzungsverzeichnis
    Allgemeines Abkürzungsverzeichnis L.
    [Show full text]
  • Active@ UNDELETE Documentation
    Active @ UNDELETE Users Guide | Contents | 2 Contents Legal Statement.........................................................................................................5 Active@ UNDELETE Overview............................................................................. 6 Getting Started with Active@ UNDELETE.......................................................... 7 Active@ UNDELETE Views And Windows...................................................................................................... 7 Recovery Explorer View.......................................................................................................................... 8 Logical Drive Scan Result View..............................................................................................................9 Physical Device Scan View......................................................................................................................9 Search Results View...............................................................................................................................11 File Organizer view................................................................................................................................ 12 Application Log...................................................................................................................................... 13 Welcome View........................................................................................................................................14 Using
    [Show full text]
  • Active @ UNDELETE Users Guide | TOC | 2
    Active @ UNDELETE Users Guide | TOC | 2 Contents Legal Statement..................................................................................................4 Active@ UNDELETE Overview............................................................................. 5 Getting Started with Active@ UNDELETE........................................................... 6 Active@ UNDELETE Views And Windows......................................................................................6 Recovery Explorer View.................................................................................................... 7 Logical Drive Scan Result View.......................................................................................... 7 Physical Device Scan View................................................................................................ 8 Search Results View........................................................................................................10 Application Log...............................................................................................................11 Welcome View................................................................................................................11 Using Active@ UNDELETE Overview................................................................. 13 Recover deleted Files and Folders.............................................................................................. 14 Scan a Volume (Logical Drive) for deleted files..................................................................15
    [Show full text]
  • Partition Wizard About Minitool Partition Wizard Minitool Partition Wizard Is an Easy-To-Use Partitioning Software with High Security and Efficiency
    MiniTool Partition Wizard About MiniTool Partition Wizard MiniTool Partition Wizard is an easy-to-use partitioning software with high security and efficiency. Due of its simple user interface, you can create, delete, format, move, and resize partitions with ease. What’s more, your data will always be protected when using MiniTool Partition Wizard to move and resize partitions. Main Functions of MiniTool Partition Wizard: Resize/ Move partitions Merge Partitions Create partitions Delete partitions Change Partition Label Delete all partitions Format partitions Change Cluster Size Convert file system Convert FAT to NTFS Convert NTFS to FAT Explore Partition Check Partitions Recovery Partition Wipe disk Wipe partition Copy partition Copy disks Initialize to MBR disk Initialize to GPT disk Align All Partitions Align Partition Convert MBR Disk to GPT Disk Convert GPT Disk to MBR Disk Dynamic Disk Create volume Delete Volume Format Volume Move/Resize Volume Wipe Volume Explore Volume Check File System Change Volume Label Change Volume Letter Change Volume Cluster Size Volume Properties MiniTool Partition Wizard Staring MiniTool Partition Wizard You can start MiniTool Partition Wizard from the Start menu in Windows Click Start menu > All Programs > MiniTool Partition Wizard xxx Edition > MiniTool Partition Wizard xxx Edition Xxx is your present edition of MiniTool Partition Wizard, Such as Home, Professional, Server, and Enterprise MiniTool Partition Wizard Hardware Requirements Minimum Hardware requirements: 500 MHz x86 or compatible CPU. 256mb RAM memory. Mouse and Keyboard. Recommended Hardware requirements: 1 GHz x86 or compatible CPU. 512mb RAM memory. Mouse and Keyboard. MiniTool Partition Wizard System Requirements Note: you should have access to administration while using Partition Wizard.
    [Show full text]
  • MSD FATFS Users Guide
    Freescale MSD FATFS Users Guide Document Number: MSDFATFSUG Rev. 0 02/2011 How to Reach Us: Home Page: www.freescale.com E-mail: [email protected] USA/Europe or Locations Not Listed: Freescale Semiconductor Technical Information Center, CH370 1300 N. Alma School Road Chandler, Arizona 85224 +1-800-521-6274 or +1-480-768-2130 [email protected] Europe, Middle East, and Africa: Information in this document is provided solely to enable system and Freescale Halbleiter Deutschland GmbH software implementers to use Freescale Semiconductor products. There are Technical Information Center no express or implied copyright licenses granted hereunder to design or Schatzbogen 7 fabricate any integrated circuits or integrated circuits based on the 81829 Muenchen, Germany information in this document. +44 1296 380 456 (English) +46 8 52200080 (English) Freescale Semiconductor reserves the right to make changes without further +49 89 92103 559 (German) notice to any products herein. Freescale Semiconductor makes no warranty, +33 1 69 35 48 48 (French) representation or guarantee regarding the suitability of its products for any particular purpose, nor does Freescale Semiconductor assume any liability [email protected] arising out of the application or use of any product or circuit, and specifically disclaims any and all liability, including without limitation consequential or Japan: incidental damages. “Typical” parameters that may be provided in Freescale Freescale Semiconductor Japan Ltd. Semiconductor data sheets and/or specifications can and do vary in different Headquarters applications and actual performance may vary over time. All operating ARCO Tower 15F parameters, including “Typicals”, must be validated for each customer 1-8-1, Shimo-Meguro, Meguro-ku, application by customer’s technical experts.
    [Show full text]
  • Computer Hardware
    Chapter Computer Hardware ENCE EXAM TOPICS COVERED IN 1 THIS CHAPTER: ✓ Computer hardware components ✓ The boot process ✓ Partitions ✓ File systems COPYRIGHTED MATERIAL Computer forensics examiners deal most often with the media on which evidentiary data is stored. This includes, but is not lim- ited to, hard drives, CDs, DVDs, fl ash memory devices, smart phones, tablets, and even legacy fl oppies and tapes. Although these devices might be the bane of the examiner’s existence, media devices don’t exist in a void, and knowledge of a computer’s various components and functions is a must for the competent examiner. As an examiner, you may be called upon to explain how a computer functions to a jury. Doing so requires you know a computer’s function from a technical standpoint and that you can translate those technical concepts into real-world, easy-to-understand terms. As an examiner, you may also be subjected to a voir dire examination by opposing coun- sel to challenge your competence to testify. Acronyms are hardly in short supply in the fi eld of computing—some well-known and meaningful, others more obscure. Imagine being asked during such an examination to explain several of the common acronyms used with computers, such as RAM, CMOS, SCSI, BIOS, and POST. If you were to draw a blank on some obscure or even common acronym, picture its impact on your credibility. Some acronyms are difficult to remember because their meaning is often obscure or meaningless. A good example is TWAIN, which stands for T ech- nology W ithout a n I nteresting N ame.
    [Show full text]
  • CIS 4360 Secure Computer Systems Attacks Against Boot And
    CIS 4360 Secure Computer Systems Attacks against Boot and RAM Professor Qiang Zeng Spring 2017 Previous Class • BIOS-MBR: Generation I system boot – What BIOS and MBR are? – How does it boot the system? // Jumping to MBR – How does multi-boot work? // Chain-loading • The limitations of BIOS and MBR – Disk, memory, file system, multi-booting, security, … • UEFI-GPT: Generation II system boot – What UEFI and GPT are? – How does it boot the system? // UEFI boot manager – How does multi-boot work? // separate dirs in ESP CIS 4360 – Secure Computer Systems 2 Limitations of BIOS-MBR • MBR is very limited – Support ~2TB disk only – 4 primary partitions at most (so four OSes at most) – A MBR can store only one boot loader • BIOS is very restrictive – 16-bit processor mode; 1MB memory space (little spare space to accommodate a file system driver) – Blindly executes whatever code on MBR CIS 4360 – Secure Computer Systems 3 UEFI vs. BIOS • Disk partitioning schemes – GPT (GUID Partition Table): part of UEFI spec.; to replace MBR – MBR supports disk size 232 x 512B = 2TB, while UEFI supports much larger disks (264 x 512B ＝ 8,000,000,000 TB) – MBR supports 4 partitions, while GPT supports 128 • Memory space – BIOS: 20-bit addressing; UEFI: 32-bit or 64-bit • Pre-OS environment – BIOS only provides raw disk access, while UEFI supports the FAT file system (so you can use file names to read files) • Booting – BIOS supports boot through boot sectors (MBR and VBR) – UEFI provides a boot partition of hundreds of megabytes (and boot manager and secure boot) CIS 4360 – Secure Computer Systems 4 Previous Class How does dual-boo-ng of Linux and Windows work in UEFI-GPT? Each vendor has a separate directory storing its own boot loader code and configuraon files in the ESP (EFI System Par--on).
    [Show full text]
  • Synchronizing Threads with POSIX Semaphores
    3/17/2016 POSIX Semaphores Synchronizing Threads with POSIX Semaphores 1. Why semaphores? 2. Posix semaphores are easy to use sem_init sem_wait sem_post sem_getvalue sem_destroy 3. Activities 1 2 Now it is time to take a look at some code that does something a little unexpected. The program badcnt.c creates two new threads, both of which increment a global variable called cnt exactly NITER, with NITER = 1,000,000. But the program produces unexpected results. Activity 1. Create a directory called posixsem in your class Unix directory. Download in this directory the code badcnt.c and compile it using gcc badcnt.c -o badcnt -lpthread Run the executable badcnt and observe the ouput. Try it on both tanner and felix. Quite unexpected! Since cnt starts at 0, and both threads increment it NITER times, we should see cnt equal to 2*NITER at the end of the program. What happens? Threads can greatly simplify writing elegant and efficient programs. However, there are problems when multiple threads share a common address space, like the variable cnt in our earlier example. To understand what might happen, let us analyze this simple piece of code: THREAD 1 THREAD 2 a = data; b = data; a++; b--; data = a; data = b; Now if this code is executed serially (for instance, THREAD 1 first and then THREAD 2), there are no problems. However threads execute in an arbitrary order, so consider the following situation: Thread 1 Thread 2 data a = data; --- 0 a = a+1; --- 0 --- b = data; // 0 0 --- b = b + 1; 0 data = a; // 1 --- 1 --- data = b; // 1 1 So data could end up +1, 0, -1, and there is NO WAY to know which value! It is completely non- deterministic! http://www.csc.villanova.edu/~mdamian/threads/posixsem.html 1/4 3/17/2016 POSIX Semaphores The solution to this is to provide functions that will block a thread if another thread is accessing data that it is using.
    [Show full text]
  • Openextensions POSIX Conformance Document
    z/VM Version 7 Release 1 OpenExtensions POSIX Conformance Document IBM GC24-6298-00 Note: Before you use this information and the product it supports, read the information in “Notices” on page 73. This edition applies to version 7, release 1, modification 0 of IBM z/VM (product number 5741-A09) and to all subsequent releases and modifications until otherwise indicated in new editions. Last updated: 2018-09-12 © Copyright International Business Machines Corporation 1993, 2018. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents List of Tables........................................................................................................ ix About This Document............................................................................................xi Intended Audience......................................................................................................................................xi Conventions Used in This Document.......................................................................................................... xi Where to Find More Information.................................................................................................................xi Links to Other Documents and Websites.............................................................................................. xi How to Send Your Comments to IBM....................................................................xiii Summary of Changes for z/VM
    [Show full text]
  • So You Think You Know C? [Pdf]
    So You Think You Know C? And Ten More Short Essays on Programming Languages by Oleksandr Kaleniuk Published in 2020 This is being published under the Creative Commons Zero license. I have dedicated this work to the public domain by waiving all of my rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law. You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. Table of Contents Introduction......................................................................................................... 4 So you think you know C?..................................................................................6 APL deserves its renaissance too.......................................................................13 Going beyond the idiomatic Python..................................................................30 Why Erlang is the only true computer language................................................39 The invisible Prolog in C++..............................................................................43 One reason you probably shouldn’t bet your whole career on JavaScript.........54 You don't have to learn assembly to read disassembly......................................57 Fortran is still a thing........................................................................................64 Learn you a Lisp in 0 minutes...........................................................................68 Blood, sweat,
    [Show full text]
  • IT Acronyms.Docx
    List of computing and IT abbreviations /.—Slashdot 1GL—First-Generation Programming Language 1NF—First Normal Form 10B2—10BASE-2 10B5—10BASE-5 10B-F—10BASE-F 10B-FB—10BASE-FB 10B-FL—10BASE-FL 10B-FP—10BASE-FP 10B-T—10BASE-T 100B-FX—100BASE-FX 100B-T—100BASE-T 100B-TX—100BASE-TX 100BVG—100BASE-VG 286—Intel 80286 processor 2B1Q—2 Binary 1 Quaternary 2GL—Second-Generation Programming Language 2NF—Second Normal Form 3GL—Third-Generation Programming Language 3NF—Third Normal Form 386—Intel 80386 processor 1 486—Intel 80486 processor 4B5BLF—4 Byte 5 Byte Local Fiber 4GL—Fourth-Generation Programming Language 4NF—Fourth Normal Form 5GL—Fifth-Generation Programming Language 5NF—Fifth Normal Form 6NF—Sixth Normal Form 8B10BLF—8 Byte 10 Byte Local Fiber A AAT—Average Access Time AA—Anti-Aliasing AAA—Authentication Authorization, Accounting AABB—Axis Aligned Bounding Box AAC—Advanced Audio Coding AAL—ATM Adaptation Layer AALC—ATM Adaptation Layer Connection AARP—AppleTalk Address Resolution Protocol ABCL—Actor-Based Concurrent Language ABI—Application Binary Interface ABM—Asynchronous Balanced Mode ABR—Area Border Router ABR—Auto Baud-Rate detection ABR—Available Bitrate 2 ABR—Average Bitrate AC—Acoustic Coupler AC—Alternating Current ACD—Automatic Call Distributor ACE—Advanced Computing Environment ACF NCP—Advanced Communications Function—Network Control Program ACID—Atomicity Consistency Isolation Durability ACK—ACKnowledgement ACK—Amsterdam Compiler Kit ACL—Access Control List ACL—Active Current
    [Show full text]