Section 9: File Systems, I/O Performance
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
File Systems and Disk Layout I/O: the Big Picture
File Systems and Disk Layout I/O: The Big Picture Processor interrupts Cache Memory Bus I/O Bridge Main I/O Bus Memory Disk Graphics Network Controller Controller Interface Disk Disk Graphics Network 1 Rotational Media Track Sector Arm Cylinder Platter Head Access time = seek time + rotational delay + transfer time seek time = 5-15 milliseconds to move the disk arm and settle on a cylinder rotational delay = 8 milliseconds for full rotation at 7200 RPM: average delay = 4 ms transfer time = 1 millisecond for an 8KB block at 8 MB/s Bandwidth utilization is less than 50% for any noncontiguous access at a block grain. Disks and Drivers Disk hardware and driver software provide basic facilities for nonvolatile secondary storage (block devices). 1. OS views the block devices as a collection of volumes. A logical volume may be a partition ofasinglediskora concatenation of multiple physical disks (e.g., RAID). 2. OS accesses each volume as an array of fixed-size sectors. Identify sector (or block) by unique (volumeID, sector ID). Read/write operations DMA data to/from physical memory. 3. Device interrupts OS on I/O completion. ISR wakes up process, updates internal records, etc. 2 Using Disk Storage Typical operating systems use disks in three different ways: 1. System calls allow user programs to access a “raw” disk. Unix: special device file identifies volume directly. Any process that can open thedevicefilecanreadorwriteany specific sector in the disk volume. 2. OS uses disk as backing storage for virtual memory. OS manages volume transparently as an “overflow area” for VM contents that do not “fit” in physical memory. -
Dealing with Document Size Limits
Dealing with Document Size Limits Introduction The Electronic Case Filing system will not accept PDF documents larger than ten megabytes (MB). If the document size is less than 10 MB, it can be filed electronically just as it is. If it is larger than 10 MB, it will need to be divided into two or more documents, with each document being less than 10 MB. Word Processing Documents Documents created with a word processing program (such as WordPerfect or Microsoft Word) and correctly converted to PDF will generally be smaller than a scanned document. Because of variances in software, usage, and content, it is difficult to estimate the number of pages that would constitute 10 MB. (Note: See “Verifying File Size” below and for larger documents, see “Splitting PDF Documents into Multiple Documents” below.) Scanned Documents Although the judges’ Filing Preferences indicate a preference for conversion of documents rather than scanning, it will be necessary to scan some documents for filing, e.g., evidentiary attachments must be scanned. Here are some things to remember: • Documents scanned to PDF are generally much larger than those converted through a word processor. • While embedded fonts may be necessary for special situations, e.g., trademark, they will increase the file size. • If graphs or color photos are included, just a few pages can easily exceed the 10 MB limit. Here are some guidelines: • The court’s standard scanner resolution is 300 dots per inch (DPI). Avoid using higher resolutions as this will create much larger file sizes. • Normally, the output should be set to black and white. -
Semiconductor Memories
Semiconductor Memories Prof. MacDonald Types of Memories! l" Volatile Memories –" require power supply to retain information –" dynamic memories l" use charge to store information and require refreshing –" static memories l" use feedback (latch) to store information – no refresh required l" Non-Volatile Memories –" ROM (Mask) –" EEPROM –" FLASH – NAND or NOR –" MRAM Memory Hierarchy! 100pS RF 100’s of bytes L1 1nS SRAM 10’s of Kbytes 10nS L2 100’s of Kbytes SRAM L3 100’s of 100nS DRAM Mbytes 1us Disks / Flash Gbytes Memory Hierarchy! l" Large memories are slow l" Fast memories are small l" Memory hierarchy gives us illusion of large memory space with speed of small memory. –" temporal locality –" spatial locality Register Files ! l" Fastest and most robust memory array l" Largest bit cell size l" Basically an array of large latches l" No sense amps – bits provide full rail data out l" Often multi-ported (i.e. 8 read ports, 2 write ports) l" Often used with ALUs in the CPU as source/destination l" Typically less than 10,000 bits –" 32 32-bit fixed point registers –" 32 60-bit floating point registers SRAM! l" Same process as logic so often combined on one die l" Smaller bit cell than register file – more dense but slower l" Uses sense amp to detect small bit cell output l" Fastest for reads and writes after register file l" Large per bit area costs –" six transistors (single port), eight transistors (dual port) l" L1 and L2 Cache on CPU is always SRAM l" On-chip Buffers – (Ethernet buffer, LCD buffer) l" Typical sizes 16k by 32 Static Memory -
EEPROM Emulation
...the world's most energy friendly microcontrollers EEPROM Emulation AN0019 - Application Note Introduction This application note demonstrates a way to use the flash memory of the EFM32 to emulate single variable rewritable EEPROM memory through software. The example API provided enables reading and writing of single variables to non-volatile flash memory. The erase-rewrite algorithm distributes page erases and thereby doing wear leveling. This application note includes: • This PDF document • Source files (zip) • Example C-code • Multiple IDE projects 2013-09-16 - an0019_Rev1.09 1 www.silabs.com ...the world's most energy friendly microcontrollers 1 General Theory 1.1 EEPROM and Flash Based Memory EEPROM stands for Electrically Erasable Programmable Read-Only Memory and is a type of non- volatile memory that is byte erasable and therefore often used to store small amounts of data that must be saved when power is removed. The EFM32 microcontrollers do not include an embedded EEPROM module for byte erasable non-volatile storage, but all EFM32s do provide flash memory for non-volatile data storage. The main difference between flash memory and EEPROM is the erasable unit size. Flash memory is block-erasable which means that bytes cannot be erased individually, instead a block consisting of several bytes need to be erased at the same time. Through software however, it is possible to emulate individually erasable rewritable byte memory using block-erasable flash memory. To provide EEPROM functionality for the EFM32s in an application, there are at least two options available. The first one is to include an external EEPROM module when designing the hardware layout of the application. -
If You Have Attempted to Upload Your Files and You Receive an Error Or
If you have attempted to upload your files and you receive an error or they simply will not upload to EDJOIN, your files are probably either in an inappropriate format, or too large. Please ensure that your document meets the appropriate format and size requirements below. Acceptable format: .PDF Size limit: Each file must not exceed 1 MB (megabyte) or 1024 KB File Name: The file name must contain less than 50 characters including spaces. If the file name contains more than 50 characters or any special characters, you may have trouble attaching your documents and/or the district you are applying to may have trouble viewing them. Please make sure your document title only contains letters and numbers. If the document is multiple pages, you may need to scan multiple sections of the document to maintain the 1MB file size allowance. Saving Your Documents to .PDF Format: If Using A PC: Microsoft Word 2007 or later allows you to save a document as .PDF on a PC 1. Open the document from where you have it saved in your computer. 2. Once the document opens in Microsoft Word, go to the File menu at the top left of your Microsoft Word window and click on Save as. 3. Note where you will be saving the document, go to the bottom of the window and select PDF (*.pdf) in the Save as type box. 4. Save and close. If Using A Mac: Using the print dialog box allows you to save just about any document to .PDF format from a Mac 1. -
Sequence Alignment/Map Format Specification
Sequence Alignment/Map Format Specification The SAM/BAM Format Specification Working Group 3 Jun 2021 The master version of this document can be found at https://github.com/samtools/hts-specs. This printing is version 53752fa from that repository, last modified on the date shown above. 1 The SAM Format Specification SAM stands for Sequence Alignment/Map format. It is a TAB-delimited text format consisting of a header section, which is optional, and an alignment section. If present, the header must be prior to the alignments. Header lines start with `@', while alignment lines do not. Each alignment line has 11 mandatory fields for essential alignment information such as mapping position, and variable number of optional fields for flexible or aligner specific information. This specification is for version 1.6 of the SAM and BAM formats. Each SAM and BAMfilemay optionally specify the version being used via the @HD VN tag. For full version history see Appendix B. Unless explicitly specified elsewhere, all fields are encoded using 7-bit US-ASCII 1 in using the POSIX / C locale. Regular expressions listed use the POSIX / IEEE Std 1003.1 extended syntax. 1.1 An example Suppose we have the following alignment with bases in lowercase clipped from the alignment. Read r001/1 and r001/2 constitute a read pair; r003 is a chimeric read; r004 represents a split alignment. Coor 12345678901234 5678901234567890123456789012345 ref AGCATGTTAGATAA**GATAGCTGTGCTAGTAGGCAGTCAGCGCCAT +r001/1 TTAGATAAAGGATA*CTG +r002 aaaAGATAA*GGATA +r003 gcctaAGCTAA +r004 ATAGCT..............TCAGC -r003 ttagctTAGGC -r001/2 CAGCGGCAT The corresponding SAM format is:2 1Charset ANSI X3.4-1968 as defined in RFC1345. -
A Hybrid Swapping Scheme Based on Per-Process Reclaim for Performance Improvement of Android Smartphones (August 2018)
Received August 19, 2018, accepted September 14, 2018, date of publication October 1, 2018, date of current version October 25, 2018. Digital Object Identifier 10.1109/ACCESS.2018.2872794 A Hybrid Swapping Scheme Based On Per-Process Reclaim for Performance Improvement of Android Smartphones (August 2018) JUNYEONG HAN 1, SUNGEUN KIM1, SUNGYOUNG LEE1, JAEHWAN LEE2, AND SUNG JO KIM2 1LG Electronics, Seoul 07336, South Korea 2School of Software, Chung-Ang University, Seoul 06974, South Korea Corresponding author: Sung Jo Kim ([email protected]) This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education under Grant 2016R1D1A1B03931004 and in part by the Chung-Ang University Research Scholarship Grants in 2015. ABSTRACT As a way to increase the actual main memory capacity of Android smartphones, most of them make use of zRAM swapping, but it has limitation in increasing its capacity since it utilizes main memory. Unfortunately, they cannot use secondary storage as a swap space due to the long response time and wear-out problem. In this paper, we propose a hybrid swapping scheme based on per-process reclaim that supports both secondary-storage swapping and zRAM swapping. It attempts to swap out all the pages in the working set of a process to a zRAM swap space rather than killing the process selected by a low-memory killer, and to swap out the least recently used pages into a secondary storage swap space. The main reason being is that frequently swap- in/out pages use the zRAM swap space while less frequently swap-in/out pages use the secondary storage swap space, in order to reduce the page operation cost. -
AN-1471 Application Note
AN-1471 APPLICATION NOTE One Technology Way • P. O. Box 9106 • Norwood, MA 02062-9106, U.S.A. • Tel: 781.329.4700 • Fax: 781.461.3113 • www.analog.com ADuCM4050 Flash EEPROM Emulation by Pranit Jadhav and Rafael Lajara INTRODUCTION provides assurance that the flash initialization function works as Nonvolatile data storage is a necessity in many embedded expected. An ECC check is enabled for the entire user space in systems. Data, such as boot up configuration, calibration constants, the flash memory of the ADuCM4050. If ECC errors are reported and network related information, is normally stored on an during any read operation, the ECC engine automatically corrects electronically erasable programmable read only memory 1-bit errors and only reports on detection of 2-bit errors. When a (EEPROM) device. The advantage of using the EEPROM to store read occurs on the flash, appropriate flags are set in the status this data is that a single byte on an EEPROM device can be register of the ADuCM4050. If interrupts are generated, the source rewritten or updated without affecting the contents in the other address of the ECC error causing an interrupt is available in the locations. FLCC0_ECC_ADDR register for the interrupt service routine (ISR) to read. The ADuCM4050 is an ultra low power microcontroller unit (MCU) with integrated flash memory. The ADuCM4050 includes Emulation of the EEPROM on the integrated flash memory 512 kB of embedded flash memory with a 72-bit wide data bus reduces the bill of material (BOM) cost by omitting the EEPROM that provides two 32-bits words of data and one corresponding in the design. -
File System Layout
File Systems Main Points • File layout • Directory layout • Reliability/durability Named Data in a File System index !le name directory !le number structure storage o"set o"set block Last Time: File System Design Constraints • For small files: – Small blocks for storage efficiency – Files used together should be stored together • For large files: – ConCguous allocaon for sequenCal access – Efficient lookup for random access • May not know at file creaon – Whether file will become small or large File System Design Opons FAT FFS NTFS Index Linked list Tree Tree structure (fixed, asym) (dynamic) granularity block block extent free space FAT array Bitmap Bitmap allocaon (fixed (file) locaon) Locality defragmentaon Block groups Extents + reserve Best fit space defrag MicrosoS File Allocaon Table (FAT) • Linked list index structure – Simple, easy to implement – SCll widely used (e.g., thumb drives) • File table: – Linear map of all blocks on disk – Each file a linked list of blocks FAT MFT Data Blocks 0 1 2 3 !le 9 block 3 4 5 6 7 8 9 !le 9 block 0 10 !le 9 block 1 11 !le 9 block 2 12 !le 12 block 0 13 14 15 16 !le 12 block 1 17 18 !le 9 block 4 19 20 FAT • Pros: – Easy to find free block – Easy to append to a file – Easy to delete a file • Cons: – Small file access is slow – Random access is very slow – Fragmentaon • File blocks for a given file may be scaered • Files in the same directory may be scaered • Problem becomes worse as disk fills Berkeley UNIX FFS (Fast File System) • inode table – Analogous to FAT table • inode – Metadata • File owner, access permissions, -
ZFS On-Disk Specification Draft
ZFS On-Disk Specification Draft Sun Microsystems, Inc. 4150 Network Circle Santa Clara, CA 95054 U.S.A 1 ©2006 Sun Microsystems, Inc. 4150 Network Circle Santa Clara, CA 95054 U.S.A. This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. Sun, Sun Microsystems, the Sun logo, Java, JavaServer Pages, Solaris, and StorEdge are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. U.S. Government Rights Commercial software. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements. DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. Unless otherwise licensed, use of this software is authorized pursuant to the terms of the license found at: http://developers.sun.com/berkeley_license.html Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l'utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l'autorisation préalable et écrite de Sun et de ses bailleurs de licence, s'il y en a. -
Mac OS X Server
Mac OS X Server Version 10.4 Technology Overview August 2006 Technology Overview 2 Mac OS X Server Contents Page 3 Introduction Page 5 New in Version 10.4 Page 7 Operating System Fundamentals UNIX-Based Foundation 64-Bit Computing Advanced BSD Networking Architecture Robust Security Directory Integration High Availability Page 10 Integrated Management Tools Server Admin Workgroup Manager Page 14 Service Deployment and Administration Open Directory Server File and Print Services Mail Services Web Hosting Enterprise Applications Media Streaming iChat Server Software Update Server NetBoot and NetInstall Networking and VPN Distributed Computing Page 29 Product Details Page 31 Open Source Projects Page 35 Additional Resources Technology Overview 3 Mac OS X Server Introduction Mac OS X Server version 10.4 Tiger gives you everything you need to manage servers in a mixed-platform environment and to con gure, deploy, and manage powerful network services. Featuring the renowned Mac OS X interface, Mac OS X Server streamlines your management tasks with applications and utilities that are robust yet easy to use. Apple’s award-winning server software brings people and data together in innovative ways. Whether you want to empower users with instant messaging and blogging, gain greater control over email, reduce the cost and hassle of updating software, or build your own distributed supercomputer, Mac OS X Server v10.4 has the tools you need. The Universal release of Mac OS X Server runs on both Intel- and PowerPC-based The power and simplicity of Mac OS X Server are a re ection of Apple’s operating sys- Mac desktop and Xserve systems. -
Converting Audio – Audio File Size
Converting Audio – Audio File Size By the end of this worksheet you should: • know how to calculate the possible file size of an audio recording You should now be very familiar with the following concepts: • BIT: a single zero (0) or (1), short for Binary digIT • BYTE: 8 bits • Analogue: a continuously varying signal • Digital: a pulse that alternates between OFF (0) and ON (1) • ADC: Analogue to Digital Converter • Sample Rate: how many times per second the ADC tests the incoming analogue signal • Sample Resolution: the number of bits allocate to each sample taken We only need to add 1 more snippet of knowledge to calculate the possible file size of an audio recording, and that is the duration of the recording. Imagine that you record 10 seconds of audio, setting the sample rate to 44.1kHz and using 16 bits per sample. What does this all mean? Page 1 of 5 Converting Audio – Audio File Size Well, we know that… • each sample uses 16 bits • there are 44 100 samples per second • this means that we are using 44 100 samples per second x 16 bits = 705 600 bits per second • Since we recorded for 10 seconds we have 705 600 bits per second x 10 seconds = 7 056 000 bits in total • There are 8 bits to a byte so 7 056 000 ÷ 8 = 882 000 bytes • 1 000 is a kilo so 882 000 bytes = 882 kilobytes or 882KB So, a 10 second recording that is set at 16 bits/44.1kHz might result in a file of 882KB.